{"thread":{"id":"65959","subject":"[PATCH 0/7] odb: introduce object filters to `odb_for_each_object()`","startedAt":"2026-07-09T08:36:02Z","lastAt":"2026-07-15T15:15:28Z","messageCount":75,"participants":["Patrick Steinhardt","Junio C Hamano","Justin Tobler","Taylor Blau","Jeff King","Toon Claes"],"isPatch":true,"patchVersion":1,"patchTotal":7},"messages":[{"id":"547572","messageId":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","threadId":"65959","inReplyTo":null,"subject":"[PATCH 0/7] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-09T08:35:20Z","receivedAt":"2026-07-09T08:36:02Z","isPatch":true,"body":"Hi,\n\nthis patch series introduces object filters to `odb_for_each_object()`.\nThe intent of this is to make `git cat-file --batch-all-objects` work\nwith pluggable object databases. Right now it doesn't because it reaches\ninto internals of the \"packed\" backend to efficiently handle bitmapped\nobjects.\n\nThe series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n2026-07-06) with ps/odb-drop-whence at 8a7ad23e11 (odb: document object\ninfo fields, 2026-07-02) merged into it.\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (7):\n      odb/source-packed: improve lookup when enumerating objects\n      pack-bitmap: mark object filter as `const`\n      pack-bitmap: allow aborting iteration of bitmapped objects\n      pack-bitmap: iterate object sources when opening bitmaps\n      pack-bitmap: introduce function to open bitmap for a single source\n      odb: introduce object filters to `odb_for_each_object()`\n      builtin/cat-file: filter objects via object database\n\n builtin/cat-file.c     |  76 +++-----------------------------\n builtin/pack-objects.c |   2 +-\n builtin/rev-list.c     |   2 +-\n odb.h                  |  12 ++++++\n odb/source-packed.c    |  77 ++++++++++++++++++++++++++++++---\n pack-bitmap.c          | 115 ++++++++++++++++++++++++++++---------------------\n pack-bitmap.h          |  10 ++++-\n 7 files changed, 164 insertions(+), 130 deletions(-)\n\n\n---\nbase-commit: 3c8e2790f2ce15e8b5d4b4e6ced711b12649f32a\nchange-id: 20260708-pks-odb-for-each-object-filter-13286fa3523d\n\n"},{"id":"547573","messageId":"20260709-pks-odb-for-each-object-filter-v1-1-82fe014b12b3@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH 1/7] odb/source-packed: improve lookup when enumerating objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-09T08:35:21Z","receivedAt":"2026-07-09T08:36:05Z","isPatch":true,"body":"When iterating through packed objects via `odb_for_each_object()` we\ndo so via two different mechanisms:\n\n  - When a multi-pack index is available we use that one to efficiently\n    loop through all objects.\n\n  - We then loop through all packfiles that aren't covered by a\n    multi-pack index.\n\nRegardless of which mechanism we use, we then iterate through all the\nobjects indexed by the respective data structure. Curiously though,\nwhile we use the indices for enumerating the objects, we completely\nignore it for the actual object lookup. Instead, we call into the\ngeneric `odb_source_read_object_info()` function, which will itself\nconsult the indices to figure out where the object in question even\nlives.\n\nThis has two consequences:\n\n  - It's inefficient, as we basically have to figure out the position of\n    the object a second time.\n\n  - It's subtly wrong, as it may now happen that a specific object will\n    be looked up via a different pack in case it exists multiple times.\n\nFix the issue by using `packed_object_info()` directly. While at it,\nrename the `store` variable to `source`.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n odb/source-packed.c | 15 ++++++++-------\n 1 file changed, 8 insertions(+), 7 deletions(-)\n\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 0edea5356d..9cfa02b7a2 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -143,7 +143,7 @@ static bool should_exclude_pack(struct packed_git *p, enum odb_for_each_object_f\n }\n \n static int for_each_prefixed_object_in_midx(\n-\tstruct odb_source_packed *store,\n+\tstruct odb_source_packed *source,\n \tstruct multi_pack_index *m,\n \tconst struct odb_for_each_object_options *opts,\n \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n@@ -170,6 +170,7 @@ static int for_each_prefixed_object_in_midx(\n \t\t */\n \t\tfor (i = first; i < num; i++) {\n \t\t\tconst struct object_id *current = NULL;\n+\t\t\tstruct packed_git *pack;\n \t\t\tstruct object_id oid;\n \n \t\t\tcurrent = nth_midxed_object_oid(&oid, m, i);\n@@ -177,9 +178,8 @@ static int for_each_prefixed_object_in_midx(\n \t\t\tif (!match_hash(len, opts->prefix->hash, current->hash))\n \t\t\t\tbreak;\n \n-\t\t\tif (opts->flags) {\n+\t\t\tif (opts->flags || data->request) {\n \t\t\t\tuint32_t pack_id = nth_midxed_pack_int_id(m, i);\n-\t\t\t\tstruct packed_git *pack;\n \n \t\t\t\tif (prepare_midx_pack(m, pack_id)) {\n \t\t\t\t\tpack_errors = true;\n@@ -193,9 +193,9 @@ static int for_each_prefixed_object_in_midx(\n \n \t\t\tif (data->request) {\n \t\t\t\tstruct object_info oi = *data->request;\n+\t\t\t\toff_t offset = nth_midxed_offset(m, i);\n \n-\t\t\t\tret = odb_source_read_object_info(&store->base, current,\n-\t\t\t\t\t\t\t\t  &oi, 0);\n+\t\t\t\tret = packed_object_info(source, pack, offset, &oi);\n \t\t\t\tif (ret)\n \t\t\t\t\tgoto out;\n \n@@ -219,7 +219,7 @@ static int for_each_prefixed_object_in_midx(\n }\n \n static int for_each_prefixed_object_in_pack(\n-\tstruct odb_source_packed *store,\n+\tstruct odb_source_packed *source,\n \tstruct packed_git *p,\n \tconst struct odb_for_each_object_options *opts,\n \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n@@ -246,8 +246,9 @@ static int for_each_prefixed_object_in_pack(\n \n \t\tif (data->request) {\n \t\t\tstruct object_info oi = *data->request;\n+\t\t\toff_t offset = nth_packed_object_offset(p, i);\n \n-\t\t\tret = odb_source_read_object_info(&store->base, &oid, &oi, 0);\n+\t\t\tret = packed_object_info(source, p, offset, &oi);\n \t\t\tif (ret)\n \t\t\t\tgoto out;\n \n\n-- \n2.55.0.175.ge4962bd3d5.dirty\n\n"},{"id":"547574","messageId":"20260709-pks-odb-for-each-object-filter-v1-2-82fe014b12b3@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH 2/7] pack-bitmap: mark object filter as `const`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-09T08:35:22Z","receivedAt":"2026-07-09T08:36:09Z","isPatch":true,"body":"The function `for_each_bitmapped_object()` accepts an optional object\nfilter. This filter is never modified by the function, but is not\ndeclared as `const`. Fix this.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 6 +++---\n pack-bitmap.h | 2 +-\n 2 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 35774b6f0c..a47c231632 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1976,7 +1976,7 @@ static void filter_bitmap_object_type(struct bitmap_index *bitmap_git,\n static int filter_bitmap(struct bitmap_index *bitmap_git,\n \t\t\t struct object_list *tip_objects,\n \t\t\t struct bitmap *to_filter,\n-\t\t\t struct list_objects_filter_options *filter)\n+\t\t\t const struct list_objects_filter_options *filter)\n {\n \tif (!filter || filter->choice == LOFC_DISABLED)\n \t\treturn 0;\n@@ -2027,7 +2027,7 @@ static int filter_bitmap(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int can_filter_bitmap(struct list_objects_filter_options *filter)\n+static int can_filter_bitmap(const struct list_objects_filter_options *filter)\n {\n \treturn !filter_bitmap(NULL, NULL, NULL, filter);\n }\n@@ -2058,7 +2058,7 @@ static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,\n }\n \n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n-\t\t\t      struct list_objects_filter_options *filter,\n+\t\t\t      const struct list_objects_filter_options *filter,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload)\n {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 19a8655457..47935eb24e 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -96,7 +96,7 @@ struct list_objects_filter_options;\n  * not supported, `0` otherwise.\n  */\n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n-\t\t\t      struct list_objects_filter_options *filter,\n+\t\t\t      const struct list_objects_filter_options *filter,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload);\n \n\n-- \n2.55.0.175.ge4962bd3d5.dirty\n\n"},{"id":"547575","messageId":"20260709-pks-odb-for-each-object-filter-v1-3-82fe014b12b3@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH 3/7] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-09T08:35:23Z","receivedAt":"2026-07-09T08:36:13Z","isPatch":true,"body":"In a subsequent commit we'll lift iteration of bitmapped objects into\nthe \"packed\" backend and make it accessible via `odb_for_each_object()`.\nThe calling convention for that function is that the callback may return\na non-zero exit code, and if so we'll abort iteration. This is currently\nimpossible to realize though, as `for_each_bitmapped_object()` will\nignore any return value and just churn through all objects completely.\n\nThis doesn't matter to the callers of `for_each_bitmapped_object()`, as\nthere's only one of them in git-cat-file(1), and the callbacks we pass\nalways return zero. But once we move the logic into the generic\ninfrastructure it becomes a latent bug waiting to happen.\n\nRefactor the code so that the return value of the `show_reach` callback\nis not ignored anymore. Instead, returning a non-zero value will cause\nus to abort iteration in both `show_objects_for_type()` and in\n`for_each_bitmapped_object()`.\n\nNote though that there's a second user of `show_objects_for_type()` with\n`traverse_bitmap_commit_list()`, and that function does indeed invoke\ncallbacks that may return non-zero. This non-zero return value never had\nany effect at all though, and the callbacks that return non-zero values\nare only ever invoked via `traverse_bitmap_commit_list()`. Consequently,\nwe adapt them to always return 0.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/pack-objects.c |  2 +-\n builtin/rev-list.c     |  2 +-\n pack-bitmap.c          | 31 +++++++++++++++++++++----------\n pack-bitmap.h          |  3 ++-\n 4 files changed, 25 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex ea5eab4cf8..8ff92c5272 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1909,7 +1909,7 @@ static int add_object_entry_from_bitmap(const struct object_id *oid,\n \t\treturn 0;\n \n \tcreate_object_entry(oid, type, name_hash, 0, 0, pack, offset);\n-\treturn 1;\n+\treturn 0;\n }\n \n struct pbase_tree_cache {\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 8f63003709..02818b81c6 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -486,7 +486,7 @@ static int show_object_fast(\n \tvoid *payload UNUSED)\n {\n \tfprintf(stdout, \"%s\\n\", oid_to_hex(oid));\n-\treturn 1;\n+\treturn 0;\n }\n \n static void print_disk_usage(off_t size)\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex a47c231632..eda38a5433 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1695,7 +1695,7 @@ static void init_type_iterator(struct ewah_or_iterator *it,\n \t}\n }\n \n-static void show_objects_for_type(\n+static int show_objects_for_type(\n \tstruct bitmap_index *bitmap_git,\n \tstruct bitmap *objects,\n \tenum object_type object_type,\n@@ -1704,6 +1704,7 @@ static void show_objects_for_type(\n {\n \tsize_t i = 0;\n \tuint32_t offset;\n+\tint ret;\n \n \tstruct ewah_or_iterator it;\n \teword_t filter;\n@@ -1749,11 +1750,17 @@ static void show_objects_for_type(\n \n \t\t\thash = bitmap_name_hash(bitmap_git, index_pos);\n \n-\t\t\tshow_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n+\t\t\tret = show_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n+\t\t\tif (ret)\n+\t\t\t\tgoto out;\n \t\t}\n \t}\n \n+\tret = 0;\n+\n+out:\n \tewah_or_iterator_release(&it);\n+\treturn ret;\n }\n \n static int in_bitmapped_pack(struct bitmap_index *bitmap_git,\n@@ -2062,6 +2069,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload)\n {\n+\tconst enum object_type types[] = {\n+\t\tOBJ_COMMIT,\n+\t\tOBJ_TREE,\n+\t\tOBJ_BLOB,\n+\t\tOBJ_TAG,\n+\t};\n \tstruct bitmap *filtered_bitmap = NULL;\n \tuint32_t objects_nr;\n \tsize_t full_word_count;\n@@ -2086,14 +2099,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\tgoto out;\n \t}\n \n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_COMMIT, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_TREE, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_BLOB, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_TAG, show_reach, payload);\n+\tfor (size_t i = 0; i < ARRAY_SIZE(types); i++) {\n+\t\tret = show_objects_for_type(bitmap_git, filtered_bitmap,\n+\t\t\t\t\t    types[i], show_reach, payload);\n+\t\tif (ret)\n+\t\t\tgoto out;\n+\t}\n \n \tret = 0;\n out:\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 47935eb24e..ae8dc491ac 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -93,7 +93,8 @@ struct list_objects_filter_options;\n /*\n  * Filter bitmapped objects and iterate through all resulting objects,\n  * executing `show_reach` for each of them. Returns `-1` in case the filter is\n- * not supported, `0` otherwise.\n+ * not supported, `0` otherwise. Aborts iteration and bubbles up the return\n+ * value in case `show_reach()` returns non-zero.\n  */\n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\t\t      const struct list_objects_filter_options *filter,\n\n-- \n2.55.0.175.ge4962bd3d5.dirty\n\n"},{"id":"547576","messageId":"20260709-pks-odb-for-each-object-filter-v1-4-82fe014b12b3@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH 4/7] pack-bitmap: iterate object sources when opening bitmaps","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-09T08:35:24Z","receivedAt":"2026-07-09T08:36:17Z","isPatch":true,"body":"When opening a bitmap for a repository we perform two steps:\n\n  - We first look for a multi-pack index bitmap in any of the object\n    sources connected to the repository.\n\n  - We then look for a packfile bitmap in any of the packfiles of any of\n    the object sources.\n\nBoth of these steps thus iterate through object sources themselves, one\nvia `odb_prepare_alternates()` and one via `repo_for_each_pack()`. This\nlayout makes it hard to introduce a way to open the bitmap of one\nspecific object source, which is functionality that we'll require in a\nsubsequent commit.\n\nReverse the loop so that we instead loop through all sources in the\nouter loop, and then for each source we try to load its bitmap via\neither the multi-pack index or via a packfile.\n\nNote that this changes the precedence of bitmaps in one specific edge\ncase: when an earlier object source only has a packfile bitmap, but a\nlater source has a multi-pack index bitmap, we now pick the packfile\nbitmap of the earlier source. Previously, a multi-pack index bitmap from\nany source would have taken precedence over all packfile bitmaps. Given\nthat object sources are ordered such that the local source comes first,\nthis arguably is an improvement, as we now prefer local bitmaps over\nbitmaps in alternates. Furthermore, we already warn about repositories\nthat have multiple bitmaps, so this setup is broken and thus arguably\nnot worth worrying about too much.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 65 ++++++++++++++++++++++++++---------------------------------\n 1 file changed, 29 insertions(+), 36 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex eda38a5433..0e3e18a557 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -680,60 +680,53 @@ static int load_bitmap(struct repository *r, struct bitmap_index *bitmap_git,\n \treturn 0;\n }\n \n-static int open_pack_bitmap(struct repository *r,\n-\t\t\t    struct bitmap_index *bitmap_git)\n+static int open_bitmap_for_source(struct odb_source_packed *source,\n+\t\t\t\t  struct bitmap_index *bitmap_git)\n {\n-\tstruct packed_git *p;\n+\tstruct multi_pack_index *midx = get_multi_pack_index(source);\n+\tstruct packfile_list_entry *e;\n \tint ret = -1;\n \n-\trepo_for_each_pack(r, p) {\n-\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n-\t\t\tret = 0;\n-\t\t\t/*\n-\t\t\t * The only reason to keep looking is to report\n-\t\t\t * duplicates.\n-\t\t\t */\n-\t\t\tif (!trace2_is_enabled())\n-\t\t\t\tbreak;\n-\t\t}\n+\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n+\t\tret = 0;\n+\n+\tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n+\t\t/*\n+\t\t * When tracing is enabled we want to keep looking to report\n+\t\t * duplicates even if we have already found a bitmap.\n+\t\t */\n+\t\tif (!ret && !trace2_is_enabled())\n+\t\t\tbreak;\n+\n+\t\tif (open_pack_bitmap_1(bitmap_git, e->pack))\n+\t\t\tcontinue;\n+\t\tret = 0;\n \t}\n \n \treturn ret;\n }\n \n-static int open_midx_bitmap(struct repository *r,\n-\t\t\t    struct bitmap_index *bitmap_git)\n+static int open_bitmap(struct repository *r,\n+\t\t       struct bitmap_index *bitmap_git)\n {\n \tstruct odb_source *source;\n-\tint ret = -1;\n+\tint found = 0;\n \n \tassert(!bitmap_git->map);\n \n \todb_prepare_alternates(r->objects);\n \tfor (source = r->objects->sources; source; source = source->next) {\n \t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\tstruct multi_pack_index *midx = get_multi_pack_index(files->packed);\n-\t\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\tret = 0;\n-\t}\n-\treturn ret;\n-}\n-\n-static int open_bitmap(struct repository *r,\n-\t\t       struct bitmap_index *bitmap_git)\n-{\n-\tint found;\n \n-\tassert(!bitmap_git->map);\n+\t\tfound |= !open_bitmap_for_source(files->packed, bitmap_git);\n \n-\tfound = !open_midx_bitmap(r, bitmap_git);\n-\n-\t/*\n-\t * these will all be skipped if we opened a midx bitmap; but run it\n-\t * anyway if tracing is enabled to report the duplicates\n-\t */\n-\tif (!found || trace2_is_enabled())\n-\t\tfound |= !open_pack_bitmap(r, bitmap_git);\n+\t\t/*\n+\t\t * The only reason to keep looking after having found a bitmap\n+\t\t * is to report duplicates.\n+\t\t */\n+\t\tif (found && !trace2_is_enabled())\n+\t\t\tbreak;\n+\t}\n \n \treturn found ? 0 : -1;\n }\n\n-- \n2.55.0.175.ge4962bd3d5.dirty\n\n"},{"id":"547577","messageId":"20260709-pks-odb-for-each-object-filter-v1-5-82fe014b12b3@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH 5/7] pack-bitmap: introduce function to open bitmap for a single source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-09T08:35:25Z","receivedAt":"2026-07-09T08:36:22Z","isPatch":true,"body":"The function `prepare_bitmap_git()` opens the first bitmap it can find\nin any of the object sources connected to the repository. In a\nsubsequent commit, the \"packed\" object database backend will learn to\nuse bitmaps to answer object filters when enumerating objects. That\nbackend operates on a single object source though, so using a bitmap\nthat potentially belongs to a different source would be wrong:\n\n  - The source would yield objects that are not part of the source\n    itself.\n\n  - The object source info would be attributed to the wrong source.\n\n  - With multiple sources, each source would enumerate the same bitmap\n    another time.\n\nIntroduce a new function `prepare_source_bitmap_git()` that only opens\nbitmaps belonging to the given object source.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 12 ++++++++++++\n pack-bitmap.h |  2 ++\n 2 files changed, 14 insertions(+)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 0e3e18a557..5d2af96e2f 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -753,6 +753,18 @@ struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx)\n \treturn NULL;\n }\n \n+struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source)\n+{\n+\tstruct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git));\n+\n+\tif (!open_bitmap_for_source(source, bitmap_git) &&\n+\t    !load_bitmap(source->base.odb->repo, bitmap_git, 0))\n+\t\treturn bitmap_git;\n+\n+\tfree_bitmap_index(bitmap_git);\n+\treturn NULL;\n+}\n+\n int bitmap_index_contains_pack(struct bitmap_index *bitmap, struct packed_git *pack)\n {\n \tfor (; bitmap; bitmap = bitmap->base) {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex ae8dc491ac..9f20fb6e56 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -9,6 +9,7 @@\n #include \"string-list.h\"\n \n struct commit;\n+struct odb_source_packed;\n struct repository;\n struct rev_info;\n \n@@ -68,6 +69,7 @@ struct bitmapped_pack {\n \n struct bitmap_index *prepare_bitmap_git(struct repository *r);\n struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx);\n+struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source);\n \n /*\n  * Given a bitmap index, determine whether it contains the pack either directly\n\n-- \n2.55.0.175.ge4962bd3d5.dirty\n\n"},{"id":"547578","messageId":"20260709-pks-odb-for-each-object-filter-v1-6-82fe014b12b3@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH 6/7] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-09T08:35:26Z","receivedAt":"2026-07-09T08:36:25Z","isPatch":true,"body":"The function `for_each_bitmapped_object()` can be used to iterate\nthrough all objects covered by a bitmap. The benefit of this function is\nthat it allows the caller to efficiently handle some object filters. For\nexample, this can be used to filter out objects of a specific type with\nsome simple bitmap operations. But callers are currently required to\nmanually wire up the use of bitmaps though, and to do so they have to\nreach into internals of a given object database source.\n\nIntroduce a new `struct odb_for_each_object_options::filter` field so\nthat the interface becomes generic. When set, then a backend may\noptionally use the filter to skip some objects that it would have\notherwise yielded.\n\nNote that the respective backends are free to ignore this field if they\ncannot meaningfully optimize for a given filter, and consequently\ncallers need to verify whether they actually want the returned objects.\nWhile annoying, we cannot easily lift this restriction anyway as the\nobject filter infrastructure supports some filters that cannot be\nanswered by the object database alone.\n\nImplement the logic for the \"packed\" source. Note that we use the new\nfunction `prepare_source_bitmap_git()` to open the bitmap: as the\nbackend operates on a single object source, we must only use bitmaps\nthat belong to that specific source. Otherwise we might yield objects\nthat are not part of the source at all, and with multiple sources we\nwould enumerate the same bitmap once per source.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n odb.h               | 12 +++++++++++\n odb/source-packed.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n pack-bitmap.c       |  3 +--\n pack-bitmap.h       |  3 +++\n 4 files changed, 78 insertions(+), 2 deletions(-)\n\ndiff --git a/odb.h b/odb.h\nindex a1e222f605..67d0b34942 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -8,6 +8,7 @@\n #include \"thread-utils.h\"\n \n struct cached_object_entry;\n+struct list_objects_filter_options;\n struct odb_source_inmemory;\n struct packed_git;\n struct repository;\n@@ -490,6 +491,17 @@ struct odb_for_each_object_options {\n \t */\n \tconst struct object_id *prefix;\n \tsize_t prefix_hex_len;\n+\n+\t/*\n+\t * Optional object filter that allows backends to skip yielding\n+\t * objects that are excluded by the filter as an optimization. The\n+\t * filter is a best-effort hint: backends may use it to skip\n+\t * excluded objects (e.g. by consulting a reachability bitmap), but\n+\t * are also free to ignore it entirely and yield every object. As a\n+\t * consequence, callers must re-apply the filter on yielded objects\n+\t * if they require strict filtering semantics.\n+\t */\n+\tconst struct list_objects_filter_options *filter;\n };\n \n /*\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 9cfa02b7a2..4777395053 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -3,11 +3,13 @@\n #include \"chdir-notify.h\"\n #include \"dir.h\"\n #include \"git-zlib.h\"\n+#include \"list-objects-filter-options.h\"\n #include \"mergesort.h\"\n #include \"midx.h\"\n #include \"odb/source-packed.h\"\n #include \"odb/streaming.h\"\n #include \"packfile.h\"\n+#include \"pack-bitmap.h\"\n \n static int find_pack_entry(struct odb_source_packed *store,\n \t\t\t   const struct object_id *oid,\n@@ -315,6 +317,37 @@ static int odb_source_packed_for_each_prefixed_object(\n \treturn ret;\n }\n \n+struct bitmapped_for_each_object_data {\n+\tstruct odb_source_packed *packed;\n+\tconst struct object_info *request;\n+\tconst struct odb_for_each_object_options *opts;\n+\todb_for_each_object_cb cb;\n+\tvoid *cb_data;\n+};\n+\n+static int bitmapped_for_each_object(const struct object_id *oid,\n+\t\t\t\t     enum object_type type UNUSED,\n+\t\t\t\t     int flags UNUSED,\n+\t\t\t\t     uint32_t hash UNUSED,\n+\t\t\t\t     struct packed_git *pack,\n+\t\t\t\t     off_t offset,\n+\t\t\t\t     void *cb_data)\n+{\n+\tstruct bitmapped_for_each_object_data *data = cb_data;\n+\n+\tif (should_exclude_pack(pack, data->opts->flags))\n+\t\treturn 0;\n+\n+\tif (data->request) {\n+\t\tstruct object_info oi = *data->request;\n+\t\tif (packed_object_info(data->packed, pack, offset, &oi) < 0)\n+\t\t\treturn -1;\n+\t\treturn data->cb(oid, &oi, data->cb_data);\n+\t}\n+\n+\treturn data->cb(oid, NULL, data->cb_data);\n+}\n+\n static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\t\t\t\t     const struct object_info *request,\n \t\t\t\t\t     odb_for_each_object_cb cb,\n@@ -328,12 +361,33 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\t.cb = cb,\n \t\t.cb_data = cb_data,\n \t};\n+\tstruct bitmap_index *bitmap = NULL;\n \tstruct packfile_list_entry *e;\n \tint pack_errors = 0, ret;\n \n \tif (opts->prefix)\n \t\treturn odb_source_packed_for_each_prefixed_object(packed, opts, &data);\n \n+\tif (opts->filter &&\n+\t    opts->filter->choice != LOFC_DISABLED &&\n+\t    can_filter_bitmap(opts->filter))\n+\t\tbitmap = prepare_bitmap_git_for_source(packed);\n+\tif (bitmap) {\n+\t\tstruct bitmapped_for_each_object_data bitmap_data = {\n+\t\t\t.packed = packed,\n+\t\t\t.request = request,\n+\t\t\t.opts = opts,\n+\t\t\t.cb = cb,\n+\t\t\t.cb_data = cb_data,\n+\t\t};\n+\n+\t\tret = for_each_bitmapped_object(bitmap, opts->filter,\n+\t\t\t\t\t\tbitmapped_for_each_object,\n+\t\t\t\t\t\t&bitmap_data);\n+\t\tif (ret)\n+\t\t\tgoto out;\n+\t}\n+\n \tpacked->skip_mru_updates = true;\n \n \tfor (e = packfile_store_get_packs(packed); e; e = e->next) {\n@@ -342,6 +396,13 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\tif (should_exclude_pack(p, opts->flags))\n \t\t\tcontinue;\n \n+\t\t/*\n+\t\t * Objects covered by the bitmap have already been yielded\n+\t\t * above; skip them here to avoid duplicates.\n+\t\t */\n+\t\tif (bitmap && bitmap_index_contains_pack(bitmap, p))\n+\t\t\tcontinue;\n+\n \t\tif (open_pack_index(p)) {\n \t\t\tpack_errors = 1;\n \t\t\tcontinue;\n@@ -357,6 +418,7 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \n out:\n \tpacked->skip_mru_updates = false;\n+\tfree_bitmap_index(bitmap);\n \n \tif (!ret && pack_errors)\n \t\tret = -1;\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 5d2af96e2f..ac9da9545f 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -2039,12 +2039,11 @@ static int filter_bitmap(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int can_filter_bitmap(const struct list_objects_filter_options *filter)\n+bool can_filter_bitmap(const struct list_objects_filter_options *filter)\n {\n \treturn !filter_bitmap(NULL, NULL, NULL, filter);\n }\n \n-\n static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,\n \t\t\t\t\t      struct bitmap *result)\n {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 9f20fb6e56..1385027c1f 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -92,6 +92,9 @@ int test_bitmap_pseudo_merge_objects(struct repository *r, uint32_t n);\n \n struct list_objects_filter_options;\n \n+/* Check whether the filter can be computed via the bitmap. */\n+bool can_filter_bitmap(const struct list_objects_filter_options *filter);\n+\n /*\n  * Filter bitmapped objects and iterate through all resulting objects,\n  * executing `show_reach` for each of them. Returns `-1` in case the filter is\n\n-- \n2.55.0.175.ge4962bd3d5.dirty\n\n"},{"id":"547579","messageId":"20260709-pks-odb-for-each-object-filter-v1-7-82fe014b12b3@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH 7/7] builtin/cat-file: filter objects via object database","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-09T08:35:27Z","receivedAt":"2026-07-09T08:36:29Z","isPatch":true,"body":"Refactor git-cat-file(1) to use the new object filter option when\nbatching all objects. This significantly simplifies the logic and\nensures that we don't have to reach into internals of the \"files\" source\nanymore.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/cat-file.c | 76 +++++-------------------------------------------------\n 1 file changed, 7 insertions(+), 69 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex b4b99a73da..1458dd76d6 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -20,7 +20,6 @@\n #include \"userdiff.h\"\n #include \"oid-array.h\"\n #include \"packfile.h\"\n-#include \"pack-bitmap.h\"\n #include \"object-file.h\"\n #include \"object-name.h\"\n #include \"odb.h\"\n@@ -844,28 +843,6 @@ static int batch_one_object_oi(const struct object_id *oid,\n \treturn payload->callback(oid, NULL, 0, payload->payload);\n }\n \n-static int batch_one_object_packed(const struct object_id *oid,\n-\t\t\t\t   struct packed_git *pack,\n-\t\t\t\t   uint32_t pos,\n-\t\t\t\t   void *_payload)\n-{\n-\tstruct for_each_object_payload *payload = _payload;\n-\treturn payload->callback(oid, pack, nth_packed_object_offset(pack, pos),\n-\t\t\t\t payload->payload);\n-}\n-\n-static int batch_one_object_bitmapped(const struct object_id *oid,\n-\t\t\t\t      enum object_type type UNUSED,\n-\t\t\t\t      int flags UNUSED,\n-\t\t\t\t      uint32_t hash UNUSED,\n-\t\t\t\t      struct packed_git *pack,\n-\t\t\t\t      off_t offset,\n-\t\t\t\t      void *_payload)\n-{\n-\tstruct for_each_object_payload *payload = _payload;\n-\treturn payload->callback(oid, pack, offset, payload->payload);\n-}\n-\n static void batch_each_object(struct batch_options *opt,\n \t\t\t      for_each_object_fn callback,\n \t\t\t      unsigned flags,\n@@ -875,56 +852,17 @@ static void batch_each_object(struct batch_options *opt,\n \t\t.callback = callback,\n \t\t.payload = _payload,\n \t};\n+\tstruct odb_source_info source_info;\n+\tstruct object_info oi = {\n+\t\t.source_infop = &source_info,\n+\t};\n \tstruct odb_for_each_object_options opts = {\n \t\t.flags = flags,\n+\t\t.filter = &opt->objects_filter,\n \t};\n-\tstruct bitmap_index *bitmap = NULL;\n-\tstruct odb_source *source;\n-\n-\t/*\n-\t * TODO: we still need to tap into implementation details of the object\n-\t * database sources. Ideally, we should extend `odb_for_each_object()`\n-\t * to handle object filters itself so that we can move the filtering\n-\t * logic into the individual sources.\n-\t */\n-\todb_prepare_alternates(the_repository->objects);\n-\tfor (source = the_repository->objects->sources; source; source = source->next) {\n-\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\tint ret = odb_source_for_each_object(&files->loose->base, NULL, batch_one_object_oi,\n-\t\t\t\t\t\t     &payload, &opts);\n-\t\tif (ret)\n-\t\t\tbreak;\n-\t}\n-\n-\tif (opt->objects_filter.choice != LOFC_DISABLED &&\n-\t    (bitmap = prepare_bitmap_git(the_repository)) &&\n-\t    !for_each_bitmapped_object(bitmap, &opt->objects_filter,\n-\t\t\t\t       batch_one_object_bitmapped, &payload)) {\n-\t\tstruct packed_git *pack;\n-\n-\t\trepo_for_each_pack(the_repository, pack) {\n-\t\t\tif (bitmap_index_contains_pack(bitmap, pack) ||\n-\t\t\t    open_pack_index(pack))\n-\t\t\t\tcontinue;\n-\t\t\tfor_each_object_in_pack(pack, batch_one_object_packed,\n-\t\t\t\t\t\t&payload, flags);\n-\t\t}\n-\t} else {\n-\t\tstruct odb_source_info source_info;\n-\t\tstruct object_info oi = {\n-\t\t\t.source_infop = &source_info,\n-\t\t};\n-\n-\t\tfor (source = the_repository->objects->sources; source; source = source->next) {\n-\t\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\t\tint ret = odb_source_for_each_object(&files->packed->base, &oi,\n-\t\t\t\t\t\t\t     batch_one_object_oi, &payload, &opts);\n-\t\t\tif (ret)\n-\t\t\t\tbreak;\n-\t\t}\n-\t}\n \n-\tfree_bitmap_index(bitmap);\n+\todb_for_each_object_ext(the_repository->objects, &oi,\n+\t\t\t\tbatch_one_object_oi, &payload, &opts);\n }\n \n static int batch_objects(struct batch_options *opt)\n\n-- \n2.55.0.175.ge4962bd3d5.dirty\n\n"},{"id":"547644","messageId":"xmqq33xsoupa.fsf@gitster.g","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-7-82fe014b12b3@pks.im","subject":"Re: [PATCH 7/7] builtin/cat-file: filter objects via object database","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-09T18:59:29Z","receivedAt":"2026-07-09T18:59:32Z","isPatch":true,"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> Refactor git-cat-file(1) to use the new object filter option when\n> batching all objects. This significantly simplifies the logic and\n> ensures that we don't have to reach into internals of the \"files\" source\n> anymore.\n\nThis would become more convincing if you spent a few lines before\npresenting the solution to give an observation of what the current\ncode does, e.g.,\n\n    When batching all objects, git-cat-file(1) reaches into the\n    internals of the object database and manually manages bitmaps to\n    apply object filters. This creates coupling between the command\n    and ODB backend internals.\n\nto highlight the perceived problem in it.  That would flow naturally\nto the description of your solution.\n\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  builtin/cat-file.c | 76 +++++-------------------------------------------------\n>  1 file changed, 7 insertions(+), 69 deletions(-)\n\nVery nice.\n\n> diff --git a/builtin/cat-file.c b/builtin/cat-file.c\n> index b4b99a73da..1458dd76d6 100644\n> --- a/builtin/cat-file.c\n> +++ b/builtin/cat-file.c\n> @@ -20,7 +20,6 @@\n>  #include \"userdiff.h\"\n>  #include \"oid-array.h\"\n>  #include \"packfile.h\"\n> -#include \"pack-bitmap.h\"\n>  #include \"object-file.h\"\n>  #include \"object-name.h\"\n>  #include \"odb.h\"\n> @@ -844,28 +843,6 @@ static int batch_one_object_oi(const struct object_id *oid,\n>  \treturn payload->callback(oid, NULL, 0, payload->payload);\n>  }\n>  \n> -static int batch_one_object_packed(const struct object_id *oid,\n> -\t\t\t\t   struct packed_git *pack,\n> -\t\t\t\t   uint32_t pos,\n> -\t\t\t\t   void *_payload)\n> -{\n> -\tstruct for_each_object_payload *payload = _payload;\n> -\treturn payload->callback(oid, pack, nth_packed_object_offset(pack, pos),\n> -\t\t\t\t payload->payload);\n> -}\n> -\n> -static int batch_one_object_bitmapped(const struct object_id *oid,\n> -\t\t\t\t      enum object_type type UNUSED,\n> -\t\t\t\t      int flags UNUSED,\n> -\t\t\t\t      uint32_t hash UNUSED,\n> -\t\t\t\t      struct packed_git *pack,\n> -\t\t\t\t      off_t offset,\n> -\t\t\t\t      void *_payload)\n> -{\n> -\tstruct for_each_object_payload *payload = _payload;\n> -\treturn payload->callback(oid, pack, offset, payload->payload);\n> -}\n> -\n>  static void batch_each_object(struct batch_options *opt,\n>  \t\t\t      for_each_object_fn callback,\n>  \t\t\t      unsigned flags,\n> @@ -875,56 +852,17 @@ static void batch_each_object(struct batch_options *opt,\n>  \t\t.callback = callback,\n>  \t\t.payload = _payload,\n>  \t};\n> +\tstruct odb_source_info source_info;\n> +\tstruct object_info oi = {\n> +\t\t.source_infop = &source_info,\n> +\t};\n>  \tstruct odb_for_each_object_options opts = {\n>  \t\t.flags = flags,\n> +\t\t.filter = &opt->objects_filter,\n>  \t};\n> -\tstruct bitmap_index *bitmap = NULL;\n> -\tstruct odb_source *source;\n> -\n> -\t/*\n> -\t * TODO: we still need to tap into implementation details of the object\n> -\t * database sources. Ideally, we should extend `odb_for_each_object()`\n> -\t * to handle object filters itself so that we can move the filtering\n> -\t * logic into the individual sources.\n> -\t */\n> -\todb_prepare_alternates(the_repository->objects);\n> -\tfor (source = the_repository->objects->sources; source; source = source->next) {\n> -\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n> -\t\tint ret = odb_source_for_each_object(&files->loose->base, NULL, batch_one_object_oi,\n> -\t\t\t\t\t\t     &payload, &opts);\n> -\t\tif (ret)\n> -\t\t\tbreak;\n> -\t}\n> -\n> -\tif (opt->objects_filter.choice != LOFC_DISABLED &&\n> -\t    (bitmap = prepare_bitmap_git(the_repository)) &&\n> -\t    !for_each_bitmapped_object(bitmap, &opt->objects_filter,\n> -\t\t\t\t       batch_one_object_bitmapped, &payload)) {\n> -\t\tstruct packed_git *pack;\n> -\n> -\t\trepo_for_each_pack(the_repository, pack) {\n> -\t\t\tif (bitmap_index_contains_pack(bitmap, pack) ||\n> -\t\t\t    open_pack_index(pack))\n> -\t\t\t\tcontinue;\n> -\t\t\tfor_each_object_in_pack(pack, batch_one_object_packed,\n> -\t\t\t\t\t\t&payload, flags);\n> -\t\t}\n> -\t} else {\n> -\t\tstruct odb_source_info source_info;\n> -\t\tstruct object_info oi = {\n> -\t\t\t.source_infop = &source_info,\n> -\t\t};\n> -\n> -\t\tfor (source = the_repository->objects->sources; source; source = source->next) {\n> -\t\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n> -\t\t\tint ret = odb_source_for_each_object(&files->packed->base, &oi,\n> -\t\t\t\t\t\t\t     batch_one_object_oi, &payload, &opts);\n> -\t\t\tif (ret)\n> -\t\t\t\tbreak;\n> -\t\t}\n> -\t}\n>  \n> -\tfree_bitmap_index(bitmap);\n> +\todb_for_each_object_ext(the_repository->objects, &oi,\n> +\t\t\t\tbatch_one_object_oi, &payload, &opts);\n>  }\n>  \n>  static int batch_objects(struct batch_options *opt)\n"},{"id":"547646","messageId":"ak_uXc0UxB_9Vk9z@denethor","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-1-82fe014b12b3@pks.im","subject":"Re: [PATCH 1/7] odb/source-packed: improve lookup when enumerating objects","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2026-07-09T19:54:17Z","receivedAt":"2026-07-09T19:54:22Z","isPatch":true,"body":"On 26/07/09 10:35AM, Patrick Steinhardt wrote:\n> When iterating through packed objects via `odb_for_each_object()` we\n> do so via two different mechanisms:\n> \n>   - When a multi-pack index is available we use that one to efficiently\n>     loop through all objects.\n> \n>   - We then loop through all packfiles that aren't covered by a\n>     multi-pack index.\n\nTo be specific, we are talking only about the for_each_object callback\nfor the packed source `odb_source_packed_for_each_object()` correct?\nAlso, this appears to only matter when we are enumerating OIDs with a\nspecific prefix.\n\n> Regardless of which mechanism we use, we then iterate through all the\n> objects indexed by the respective data structure. Curiously though,\n> while we use the indices for enumerating the objects, we completely\n> ignore it for the actual object lookup. Instead, we call into the\n> generic `odb_source_read_object_info()` function, which will itself\n> consult the indices to figure out where the object in question even\n> lives.\n> \n> This has two consequences:\n> \n>   - It's inefficient, as we basically have to figure out the position of\n>     the object a second time.\n\nSince we already have the position from the index, there is no need to\nstart over. Makes sense.\n\n>   - It's subtly wrong, as it may now happen that a specific object will\n>     be looked up via a different pack in case it exists multiple times.\n\nNaive question: Is there any real harm in reading the same object, but\nfrom a different packfile here?\n\nRegardless I do think it's a good idea to just reuse the same packfile\nto get the same object here.\n\n> Fix the issue by using `packed_object_info()` directly. While at it,\n> rename the `store` variable to `source`.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  odb/source-packed.c | 15 ++++++++-------\n>  1 file changed, 8 insertions(+), 7 deletions(-)\n> \n> diff --git a/odb/source-packed.c b/odb/source-packed.c\n> index 0edea5356d..9cfa02b7a2 100644\n> --- a/odb/source-packed.c\n> +++ b/odb/source-packed.c\n> @@ -143,7 +143,7 @@ static bool should_exclude_pack(struct packed_git *p, enum odb_for_each_object_f\n>  }\n>  \n>  static int for_each_prefixed_object_in_midx(\n> -\tstruct odb_source_packed *store,\n> +\tstruct odb_source_packed *source,\n>  \tstruct multi_pack_index *m,\n>  \tconst struct odb_for_each_object_options *opts,\n>  \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n> @@ -170,6 +170,7 @@ static int for_each_prefixed_object_in_midx(\n>  \t\t */\n>  \t\tfor (i = first; i < num; i++) {\n>  \t\t\tconst struct object_id *current = NULL;\n> +\t\t\tstruct packed_git *pack;\n>  \t\t\tstruct object_id oid;\n>  \n>  \t\t\tcurrent = nth_midxed_object_oid(&oid, m, i);\n> @@ -177,9 +178,8 @@ static int for_each_prefixed_object_in_midx(\n>  \t\t\tif (!match_hash(len, opts->prefix->hash, current->hash))\n>  \t\t\t\tbreak;\n>  \n> -\t\t\tif (opts->flags) {\n> +\t\t\tif (opts->flags || data->request) {\n\nI'm not sure I follow why the above condition needed to change.\n\n>  \t\t\t\tuint32_t pack_id = nth_midxed_pack_int_id(m, i);\n> -\t\t\t\tstruct packed_git *pack;\n>  \n>  \t\t\t\tif (prepare_midx_pack(m, pack_id)) {\n>  \t\t\t\t\tpack_errors = true;\n> @@ -193,9 +193,9 @@ static int for_each_prefixed_object_in_midx(\n>  \n>  \t\t\tif (data->request) {\n>  \t\t\t\tstruct object_info oi = *data->request;\n> +\t\t\t\toff_t offset = nth_midxed_offset(m, i);\n>  \n> -\t\t\t\tret = odb_source_read_object_info(&store->base, current,\n> -\t\t\t\t\t\t\t\t  &oi, 0);\n> +\t\t\t\tret = packed_object_info(source, pack, offset, &oi);\n\nWe not longer use the generic function to read object info. This ensures\nthe exact same object is read.\n\n>  \t\t\t\tif (ret)\n>  \t\t\t\t\tgoto out;\n>  \n> @@ -219,7 +219,7 @@ static int for_each_prefixed_object_in_midx(\n>  }\n>  \n>  static int for_each_prefixed_object_in_pack(\n> -\tstruct odb_source_packed *store,\n> +\tstruct odb_source_packed *source,\n>  \tstruct packed_git *p,\n>  \tconst struct odb_for_each_object_options *opts,\n>  \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n> @@ -246,8 +246,9 @@ static int for_each_prefixed_object_in_pack(\n>  \n>  \t\tif (data->request) {\n>  \t\t\tstruct object_info oi = *data->request;\n> +\t\t\toff_t offset = nth_packed_object_offset(p, i);\n>  \n> -\t\t\tret = odb_source_read_object_info(&store->base, &oid, &oi, 0);\n> +\t\t\tret = packed_object_info(source, p, offset, &oi);\n\nAnd we do the same thing here when reading the object from a packfile.\n\n-Justin\n"},{"id":"547648","messageId":"alAAN6_ZqLj9tlgV@denethor","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-3-82fe014b12b3@pks.im","subject":"Re: [PATCH 3/7] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2026-07-09T20:19:52Z","receivedAt":"2026-07-09T20:19:56Z","isPatch":true,"body":"On 26/07/09 10:35AM, Patrick Steinhardt wrote:\n> In a subsequent commit we'll lift iteration of bitmapped objects into\n> the \"packed\" backend and make it accessible via `odb_for_each_object()`.\n> The calling convention for that function is that the callback may return\n> a non-zero exit code, and if so we'll abort iteration. This is currently\n> impossible to realize though, as `for_each_bitmapped_object()` will\n> ignore any return value and just churn through all objects completely.\n\nOk.\n\n> This doesn't matter to the callers of `for_each_bitmapped_object()`, as\n> there's only one of them in git-cat-file(1), and the callbacks we pass\n> always return zero. But once we move the logic into the generic\n> infrastructure it becomes a latent bug waiting to happen.\n> \n> Refactor the code so that the return value of the `show_reach` callback\n> is not ignored anymore. Instead, returning a non-zero value will cause\n> us to abort iteration in both `show_objects_for_type()` and in\n> `for_each_bitmapped_object()`.\n\nMake sense. We want to ensure that the `show_reach` callback can\nproperly signal back to `for_each_bitmapped_object()` to abort.\n\n> Note though that there's a second user of `show_objects_for_type()` with\n> `traverse_bitmap_commit_list()`, and that function does indeed invoke\n> callbacks that may return non-zero. This non-zero return value never had\n> any effect at all though, and the callbacks that return non-zero values\n> are only ever invoked via `traverse_bitmap_commit_list()`. Consequently,\n> we adapt them to always return 0.\n> \n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  builtin/pack-objects.c |  2 +-\n>  builtin/rev-list.c     |  2 +-\n>  pack-bitmap.c          | 31 +++++++++++++++++++++----------\n>  pack-bitmap.h          |  3 ++-\n>  4 files changed, 25 insertions(+), 13 deletions(-)\n> \n> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> index ea5eab4cf8..8ff92c5272 100644\n> --- a/builtin/pack-objects.c\n> +++ b/builtin/pack-objects.c\n> @@ -1909,7 +1909,7 @@ static int add_object_entry_from_bitmap(const struct object_id *oid,\n>  \t\treturn 0;\n>  \n>  \tcreate_object_entry(oid, type, name_hash, 0, 0, pack, offset);\n> -\treturn 1;\n> +\treturn 0;\n\nI wonder why this was even returning 1 to begin with? As you mentioned,\nthe return value appears to be ignored anyways. I'm assuming it was\nsignal that an object entry was created?\n\n>  }\n>  \n>  struct pbase_tree_cache {\n> diff --git a/builtin/rev-list.c b/builtin/rev-list.c\n> index 8f63003709..02818b81c6 100644\n> --- a/builtin/rev-list.c\n> +++ b/builtin/rev-list.c\n> @@ -486,7 +486,7 @@ static int show_object_fast(\n>  \tvoid *payload UNUSED)\n>  {\n>  \tfprintf(stdout, \"%s\\n\", oid_to_hex(oid));\n> -\treturn 1;\n> +\treturn 0;\n\nAlso curious about this one too. It probably doesn't matter though.\n\n>  }\n>  \n>  static void print_disk_usage(off_t size)\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index a47c231632..eda38a5433 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -1695,7 +1695,7 @@ static void init_type_iterator(struct ewah_or_iterator *it,\n>  \t}\n>  }\n>  \n> -static void show_objects_for_type(\n> +static int show_objects_for_type(\n>  \tstruct bitmap_index *bitmap_git,\n>  \tstruct bitmap *objects,\n>  \tenum object_type object_type,\n> @@ -1704,6 +1704,7 @@ static void show_objects_for_type(\n>  {\n>  \tsize_t i = 0;\n>  \tuint32_t offset;\n> +\tint ret;\n>  \n>  \tstruct ewah_or_iterator it;\n>  \teword_t filter;\n> @@ -1749,11 +1750,17 @@ static void show_objects_for_type(\n>  \n>  \t\t\thash = bitmap_name_hash(bitmap_git, index_pos);\n>  \n> -\t\t\tshow_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n> +\t\t\tret = show_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n> +\t\t\tif (ret)\n> +\t\t\t\tgoto out;\n\nThe show_reach callback now wires back its return code.\n\n>  \t\t}\n>  \t}\n>  \n> +\tret = 0;\n> +\n> +out:\n>  \tewah_or_iterator_release(&it);\n> +\treturn ret;\n>  }\n>  \n>  static int in_bitmapped_pack(struct bitmap_index *bitmap_git,\n> @@ -2062,6 +2069,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n>  \t\t\t      show_reachable_fn show_reach,\n>  \t\t\t      void *payload)\n>  {\n> +\tconst enum object_type types[] = {\n> +\t\tOBJ_COMMIT,\n> +\t\tOBJ_TREE,\n> +\t\tOBJ_BLOB,\n> +\t\tOBJ_TAG,\n> +\t};\n>  \tstruct bitmap *filtered_bitmap = NULL;\n>  \tuint32_t objects_nr;\n>  \tsize_t full_word_count;\n> @@ -2086,14 +2099,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n>  \t\tgoto out;\n>  \t}\n>  \n> -\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n> -\t\t\t      OBJ_COMMIT, show_reach, payload);\n> -\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n> -\t\t\t      OBJ_TREE, show_reach, payload);\n> -\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n> -\t\t\t      OBJ_BLOB, show_reach, payload);\n> -\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n> -\t\t\t      OBJ_TAG, show_reach, payload);\n> +\tfor (size_t i = 0; i < ARRAY_SIZE(types); i++) {\n> +\t\tret = show_objects_for_type(bitmap_git, filtered_bitmap,\n> +\t\t\t\t\t    types[i], show_reach, payload);\n> +\t\tif (ret)\n> +\t\t\tgoto out;\n> +\t}\n\n`for_each_bitmapped_object()` now has access to the underlying return\ncode and can abort. Looks good.\n\n-Justin\n"},{"id":"547651","messageId":"alADU8qRZcPB0Zcv@denethor","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-4-82fe014b12b3@pks.im","subject":"Re: [PATCH 4/7] pack-bitmap: iterate object sources when opening bitmaps","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2026-07-09T21:08:31Z","receivedAt":"2026-07-09T21:08:36Z","isPatch":true,"body":"On 26/07/09 10:35AM, Patrick Steinhardt wrote:\n> When opening a bitmap for a repository we perform two steps:\n> \n>   - We first look for a multi-pack index bitmap in any of the object\n>     sources connected to the repository.\n> \n>   - We then look for a packfile bitmap in any of the packfiles of any of\n>     the object sources.\n\nSo IIUC, we generally stop searching for a bitmap once we find one.\n\n> Both of these steps thus iterate through object sources themselves, one\n> via `odb_prepare_alternates()` and one via `repo_for_each_pack()`. This\n> layout makes it hard to introduce a way to open the bitmap of one\n> specific object source, which is functionality that we'll require in a\n> subsequent commit.\n> \n> Reverse the loop so that we instead loop through all sources in the\n> outer loop, and then for each source we try to load its bitmap via\n> either the multi-pack index or via a packfile.\n\nConceptually, I think this is a lot easier to follow too which is nice.\n\n> Note that this changes the precedence of bitmaps in one specific edge\n> case: when an earlier object source only has a packfile bitmap, but a\n> later source has a multi-pack index bitmap, we now pick the packfile\n> bitmap of the earlier source. Previously, a multi-pack index bitmap from\n> any source would have taken precedence over all packfile bitmaps. Given\n> that object sources are ordered such that the local source comes first,\n> this arguably is an improvement, as we now prefer local bitmaps over\n> bitmaps in alternates. Furthermore, we already warn about repositories\n> that have multiple bitmaps, so this setup is broken and thus arguably\n> not worth worrying about too much.\n\nI agree that the change in bitmap precedent is probably not a big deal.\nHaving multiple bitmaps in a repository is already something we warn\nagainst so I think this should be fine.\n\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  pack-bitmap.c | 65 ++++++++++++++++++++++++++---------------------------------\n>  1 file changed, 29 insertions(+), 36 deletions(-)\n> \n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index eda38a5433..0e3e18a557 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -680,60 +680,53 @@ static int load_bitmap(struct repository *r, struct bitmap_index *bitmap_git,\n>  \treturn 0;\n>  }\n>  \n> -static int open_pack_bitmap(struct repository *r,\n> -\t\t\t    struct bitmap_index *bitmap_git)\n> +static int open_bitmap_for_source(struct odb_source_packed *source,\n> +\t\t\t\t  struct bitmap_index *bitmap_git)\n>  {\n> -\tstruct packed_git *p;\n> +\tstruct multi_pack_index *midx = get_multi_pack_index(source);\n> +\tstruct packfile_list_entry *e;\n>  \tint ret = -1;\n>  \n> -\trepo_for_each_pack(r, p) {\n> -\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n> -\t\t\tret = 0;\n> -\t\t\t/*\n> -\t\t\t * The only reason to keep looking is to report\n> -\t\t\t * duplicates.\n> -\t\t\t */\n> -\t\t\tif (!trace2_is_enabled())\n> -\t\t\t\tbreak;\n> -\t\t}\n> +\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n> +\t\tret = 0;\n\nOk, open_midx_bitmap_1() returns 0 if it find a MIDX and -1 otherwise.\nProbably just a matter of preference, but I think writing out like below\nis a little bit easier on the eyes:\n\n  if (midx)\n    ret = open_midx_bitmap_1(bitmap_git, midx);\n\nit might just be that I find the return values a bit confusing though.\nMaybe we could instead use `found` like a bit later in this patch.\n\n> +\n> +\tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n> +\t\t/*\n> +\t\t * When tracing is enabled we want to keep looking to report\n> +\t\t * duplicates even if we have already found a bitmap.\n> +\t\t */\n> +\t\tif (!ret && !trace2_is_enabled())\n> +\t\t\tbreak;\n\nSo if have already found a bitmap from the MIDX and tracing is not\nenabled, we don't continue searching for bitmaps in this source. \n\n> +\n> +\t\tif (open_pack_bitmap_1(bitmap_git, e->pack))\n> +\t\t\tcontinue;\n> +\t\tret = 0;\n>  \t}\n>  \n>  \treturn ret;\n>  }\n>  \n> -static int open_midx_bitmap(struct repository *r,\n> -\t\t\t    struct bitmap_index *bitmap_git)\n> +static int open_bitmap(struct repository *r,\n> +\t\t       struct bitmap_index *bitmap_git)\n>  {\n>  \tstruct odb_source *source;\n> -\tint ret = -1;\n> +\tint found = 0;\n>  \n>  \tassert(!bitmap_git->map);\n>  \n>  \todb_prepare_alternates(r->objects);\n>  \tfor (source = r->objects->sources; source; source = source->next) {\n>  \t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n> -\t\tstruct multi_pack_index *midx = get_multi_pack_index(files->packed);\n> -\t\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n> -\t\t\tret = 0;\n> -\t}\n> -\treturn ret;\n> -}\n> -\n> -static int open_bitmap(struct repository *r,\n> -\t\t       struct bitmap_index *bitmap_git)\n> -{\n> -\tint found;\n>  \n> -\tassert(!bitmap_git->map);\n> +\t\tfound |= !open_bitmap_for_source(files->packed, bitmap_git);\n>  \n> -\tfound = !open_midx_bitmap(r, bitmap_git);\n> -\n> -\t/*\n> -\t * these will all be skipped if we opened a midx bitmap; but run it\n> -\t * anyway if tracing is enabled to report the duplicates\n> -\t */\n> -\tif (!found || trace2_is_enabled())\n> -\t\tfound |= !open_pack_bitmap(r, bitmap_git);\n> +\t\t/*\n> +\t\t * The only reason to keep looking after having found a bitmap\n> +\t\t * is to report duplicates.\n> +\t\t */\n> +\t\tif (found && !trace2_is_enabled())\n> +\t\t\tbreak;\n> +\t}\n\nOk, we only advance to the next source if tracing is enabled to print\nwarnings for multiple bitmaps. Makes sense.\n\nOverall I quite like the direction of this patch.\n\n-Justin\n"},{"id":"547652","messageId":"alATd_YS2d_l3CHq@denethor","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-6-82fe014b12b3@pks.im","subject":"Re: [PATCH 6/7] odb: introduce object filters to `odb_for_each_object()`","fromName":"Justin Tobler","fromEmail":"jltobler@gmail.com","sentAt":"2026-07-09T21:43:58Z","receivedAt":"2026-07-09T21:44:03Z","isPatch":true,"body":"On 26/07/09 10:35AM, Patrick Steinhardt wrote:\n> The function `for_each_bitmapped_object()` can be used to iterate\n> through all objects covered by a bitmap. The benefit of this function is\n> that it allows the caller to efficiently handle some object filters. For\n> example, this can be used to filter out objects of a specific type with\n> some simple bitmap operations. But callers are currently required to\n> manually wire up the use of bitmaps though, and to do so they have to\n> reach into internals of a given object database source.\n> \n> Introduce a new `struct odb_for_each_object_options::filter` field so\n> that the interface becomes generic. When set, then a backend may\n> optionally use the filter to skip some objects that it would have\n> otherwise yielded.\n> \n> Note that the respective backends are free to ignore this field if they\n> cannot meaningfully optimize for a given filter, and consequently\n> callers need to verify whether they actually want the returned objects.\n> While annoying, we cannot easily lift this restriction anyway as the\n> object filter infrastructure supports some filters that cannot be\n> answered by the object database alone.\n\nHuh, this feels rather awkward. So callers will always still have to\nensure correctness by filtering the result a second time? IIUC, the idea\nis that the backend may be able to more efficiently process object\nfiltering so we would want it to attempt the first pass.\n\nIs there a subset of object filters that we should expect any backend to\nbe able to answer? If so, maybe we should define a separate list of\nobject filter options specific to this interface? Any filtering not\nsupported would have to be deligated to the caller then.\n\n-Justin\n"},{"id":"547680","messageId":"alCacqAl9gItMKo7@pks.im","threadId":"65959","inReplyTo":"ak_uXc0UxB_9Vk9z@denethor","subject":"Re: [PATCH 1/7] odb/source-packed: improve lookup when enumerating objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T07:08:34Z","receivedAt":"2026-07-10T07:08:45Z","isPatch":true,"body":"On Thu, Jul 09, 2026 at 02:54:17PM -0500, Justin Tobler wrote:\n> On 26/07/09 10:35AM, Patrick Steinhardt wrote:\n> > When iterating through packed objects via `odb_for_each_object()` we\n> > do so via two different mechanisms:\n> > \n> >   - When a multi-pack index is available we use that one to efficiently\n> >     loop through all objects.\n> > \n> >   - We then loop through all packfiles that aren't covered by a\n> >     multi-pack index.\n> \n> To be specific, we are talking only about the for_each_object callback\n> for the packed source `odb_source_packed_for_each_object()` correct?\n> Also, this appears to only matter when we are enumerating OIDs with a\n> specific prefix.\n\nYeah, true. I'll clarify this a bit.\n\n> > Regardless of which mechanism we use, we then iterate through all the\n> > objects indexed by the respective data structure. Curiously though,\n> > while we use the indices for enumerating the objects, we completely\n> > ignore it for the actual object lookup. Instead, we call into the\n> > generic `odb_source_read_object_info()` function, which will itself\n> > consult the indices to figure out where the object in question even\n> > lives.\n> > \n> > This has two consequences:\n> > \n> >   - It's inefficient, as we basically have to figure out the position of\n> >     the object a second time.\n> \n> Since we already have the position from the index, there is no need to\n> start over. Makes sense.\n> \n> >   - It's subtly wrong, as it may now happen that a specific object will\n> >     be looked up via a different pack in case it exists multiple times.\n> \n> Naive question: Is there any real harm in reading the same object, but\n> from a different packfile here?\n\nThe answer is probably \"no\". At least I cannot think of any case where\nit'd really matter, but semantically it's the wrong thing to do anyway.\n\n> > diff --git a/odb/source-packed.c b/odb/source-packed.c\n> > index 0edea5356d..9cfa02b7a2 100644\n> > --- a/odb/source-packed.c\n> > +++ b/odb/source-packed.c\n> > @@ -177,9 +178,8 @@ static int for_each_prefixed_object_in_midx(\n> >  \t\t\tif (!match_hash(len, opts->prefix->hash, current->hash))\n> >  \t\t\t\tbreak;\n> >  \n> > -\t\t\tif (opts->flags) {\n> > +\t\t\tif (opts->flags || data->request) {\n> \n> I'm not sure I follow why the above condition needed to change.\n\nThis needs to change because we now require access to the pack so that\nwe can call `packed_object_info()`. Otherwise the pack would be not be\npopulated if we're invoked without any flags.\n\nPatrick\n"},{"id":"547681","messageId":"alCafO91ZtFdikPg@pks.im","threadId":"65959","inReplyTo":"alAAN6_ZqLj9tlgV@denethor","subject":"Re: [PATCH 3/7] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T07:08:44Z","receivedAt":"2026-07-10T07:08:48Z","isPatch":true,"body":"On Thu, Jul 09, 2026 at 03:19:52PM -0500, Justin Tobler wrote:\n> > diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> > index ea5eab4cf8..8ff92c5272 100644\n> > --- a/builtin/pack-objects.c\n> > +++ b/builtin/pack-objects.c\n> > @@ -1909,7 +1909,7 @@ static int add_object_entry_from_bitmap(const struct object_id *oid,\n> >  \t\treturn 0;\n> >  \n> >  \tcreate_object_entry(oid, type, name_hash, 0, 0, pack, offset);\n> > -\treturn 1;\n> > +\treturn 0;\n> \n> I wonder why this was even returning 1 to begin with? As you mentioned,\n> the return value appears to be ignored anyways. I'm assuming it was\n> signal that an object entry was created?\n\nThe function is only called from a single location, and the return value\nwas completely ignored until this commit. It has always been this way\nsince the function was originally introduced in 6b8fda2db1\n(pack-objects: use bitmaps when packing objects, 2013-12-21), so it\nnever seemed to have any purpose. The commit message doesn't mention\nanything either.\n\n> > diff --git a/builtin/rev-list.c b/builtin/rev-list.c\n> > index 8f63003709..02818b81c6 100644\n> > --- a/builtin/rev-list.c\n> > +++ b/builtin/rev-list.c\n> > @@ -486,7 +486,7 @@ static int show_object_fast(\n> >  \tvoid *payload UNUSED)\n> >  {\n> >  \tfprintf(stdout, \"%s\\n\", oid_to_hex(oid));\n> > -\treturn 1;\n> > +\treturn 0;\n> \n> Also curious about this one too. It probably doesn't matter though.\n\nLikewise, this was introduced in aa32939fea (rev-list: add bitmap mode\nto speed up object lists, 2013-12-21), and the return value wasn't ever\nused for anything.\n\nPatrick\n"},{"id":"547682","messageId":"alCaggWOOoDMgI09@pks.im","threadId":"65959","inReplyTo":"alADU8qRZcPB0Zcv@denethor","subject":"Re: [PATCH 4/7] pack-bitmap: iterate object sources when opening bitmaps","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T07:08:50Z","receivedAt":"2026-07-10T07:08:54Z","isPatch":true,"body":"On Thu, Jul 09, 2026 at 04:08:31PM -0500, Justin Tobler wrote:\n> On 26/07/09 10:35AM, Patrick Steinhardt wrote:\n> > When opening a bitmap for a repository we perform two steps:\n> > \n> >   - We first look for a multi-pack index bitmap in any of the object\n> >     sources connected to the repository.\n> > \n> >   - We then look for a packfile bitmap in any of the packfiles of any of\n> >     the object sources.\n> \n> So IIUC, we generally stop searching for a bitmap once we find one.\n\nExcept that we continue searching so that we can print a warning, but\nlater results are simply being ignored.\n\n> > diff --git a/pack-bitmap.c b/pack-bitmap.c\n> > index eda38a5433..0e3e18a557 100644\n> > --- a/pack-bitmap.c\n> > +++ b/pack-bitmap.c\n> > @@ -680,60 +680,53 @@ static int load_bitmap(struct repository *r, struct bitmap_index *bitmap_git,\n> >  \treturn 0;\n> >  }\n> >  \n> > -static int open_pack_bitmap(struct repository *r,\n> > -\t\t\t    struct bitmap_index *bitmap_git)\n> > +static int open_bitmap_for_source(struct odb_source_packed *source,\n> > +\t\t\t\t  struct bitmap_index *bitmap_git)\n> >  {\n> > -\tstruct packed_git *p;\n> > +\tstruct multi_pack_index *midx = get_multi_pack_index(source);\n> > +\tstruct packfile_list_entry *e;\n> >  \tint ret = -1;\n> >  \n> > -\trepo_for_each_pack(r, p) {\n> > -\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n> > -\t\t\tret = 0;\n> > -\t\t\t/*\n> > -\t\t\t * The only reason to keep looking is to report\n> > -\t\t\t * duplicates.\n> > -\t\t\t */\n> > -\t\t\tif (!trace2_is_enabled())\n> > -\t\t\t\tbreak;\n> > -\t\t}\n> > +\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n> > +\t\tret = 0;\n> \n> Ok, open_midx_bitmap_1() returns 0 if it find a MIDX and -1 otherwise.\n> Probably just a matter of preference, but I think writing out like below\n> is a little bit easier on the eyes:\n> \n>   if (midx)\n>     ret = open_midx_bitmap_1(bitmap_git, midx);\n> \n> it might just be that I find the return values a bit confusing though.\n> Maybe we could instead use `found` like a bit later in this patch.\n\nThat's fair, and the result would be both easier to reason about and\nmore consistent indeed. I'll adapt this accordingly.\n\nPatrick\n"},{"id":"547683","messageId":"alCakwxxOc4FEEAv@pks.im","threadId":"65959","inReplyTo":"alATd_YS2d_l3CHq@denethor","subject":"Re: [PATCH 6/7] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T07:09:07Z","receivedAt":"2026-07-10T07:09:12Z","isPatch":true,"body":"On Thu, Jul 09, 2026 at 04:43:58PM -0500, Justin Tobler wrote:\n> On 26/07/09 10:35AM, Patrick Steinhardt wrote:\n> > The function `for_each_bitmapped_object()` can be used to iterate\n> > through all objects covered by a bitmap. The benefit of this function is\n> > that it allows the caller to efficiently handle some object filters. For\n> > example, this can be used to filter out objects of a specific type with\n> > some simple bitmap operations. But callers are currently required to\n> > manually wire up the use of bitmaps though, and to do so they have to\n> > reach into internals of a given object database source.\n> > \n> > Introduce a new `struct odb_for_each_object_options::filter` field so\n> > that the interface becomes generic. When set, then a backend may\n> > optionally use the filter to skip some objects that it would have\n> > otherwise yielded.\n> > \n> > Note that the respective backends are free to ignore this field if they\n> > cannot meaningfully optimize for a given filter, and consequently\n> > callers need to verify whether they actually want the returned objects.\n> > While annoying, we cannot easily lift this restriction anyway as the\n> > object filter infrastructure supports some filters that cannot be\n> > answered by the object database alone.\n> \n> Huh, this feels rather awkward. So callers will always still have to\n> ensure correctness by filtering the result a second time? IIUC, the idea\n> is that the backend may be able to more efficiently process object\n> filtering so we would want it to attempt the first pass.\n> \n> Is there a subset of object filters that we should expect any backend to\n> be able to answer? If so, maybe we should define a separate list of\n> object filter options specific to this interface? Any filtering not\n> supported would have to be deligated to the caller then.\n\nIt's a bit awkward, but it's also similar to how we handle this for\nexample in the reference backends with the exclude patterns. I don't\nreally think it makes sense to enforce that backends may only handle a\nsubset of object filters that we know the current backends support, as\nthat would artificially limit us.\n\nFor example, the \"loose\" backend already cannot efficiently handle many\nof the filters that the \"packed\" backend can handle, like for example\nfiltering by type. So ultimately, the subset of filters that can be\nhandled efficiently by both backends is empty. And as the \"files\"\nbackend always combines both of these backends we wouldn't be able to\never use the object filter at all there.\n\nThe same could be true for any future backend: we cannot assume how they\nstore their objects, so they might be able to efficiently handle filters\nthat the current backends cannot.\n\nAn alternative going forward could be to perform filtering of yielded\nobjects inside `odb_for_each_object()` itself so that it will filter out\nany objects that the backends themselves couldn't filter efficiently.\nBut I'm not sure I want to go there as part of this series -- we only\nhave a single caller anyway that iterates with a filter, and that caller\nalready knows to manually filter references.\n\nI'll add a bit of an explanation to the commit message.\n\nPatrick\n"},{"id":"547684","messageId":"alCamGRrhBJgvMKQ@pks.im","threadId":"65959","inReplyTo":"xmqq33xsoupa.fsf@gitster.g","subject":"Re: [PATCH 7/7] builtin/cat-file: filter objects via object database","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T07:09:12Z","receivedAt":"2026-07-10T07:09:17Z","isPatch":true,"body":"On Thu, Jul 09, 2026 at 11:59:29AM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > Refactor git-cat-file(1) to use the new object filter option when\n> > batching all objects. This significantly simplifies the logic and\n> > ensures that we don't have to reach into internals of the \"files\" source\n> > anymore.\n> \n> This would become more convincing if you spent a few lines before\n> presenting the solution to give an observation of what the current\n> code does, e.g.,\n> \n>     When batching all objects, git-cat-file(1) reaches into the\n>     internals of the object database and manually manages bitmaps to\n>     apply object filters. This creates coupling between the command\n>     and ODB backend internals.\n> \n> to highlight the perceived problem in it.  That would flow naturally\n> to the description of your solution.\n\nGood point, will add.\n\nPatrick\n"},{"id":"547688","messageId":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH v2 0/8] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:48:52Z","receivedAt":"2026-07-10T08:49:04Z","isPatch":true,"body":"Hi,\n\nthis patch series introduces object filters to `odb_for_each_object()`.\nThe intent of this is to make `git cat-file --batch-all-objects` work\nwith pluggable object databases. Right now it doesn't because it reaches\ninto internals of the \"packed\" backend to efficiently handle bitmapped\nobjects.\n\nThe series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n2026-07-06) with ps/odb-drop-whence at 8a7ad23e11 (odb: document object\ninfo fields, 2026-07-02) merged into it.\n\nChanges in v2:\n  - Add another patch to drop the `_1()` prefixes that aren't required\n    anymore.\n  - Change the approach in `open_bitmap_for_source()` to also use a\n    `found` boolean instead of a confusing integer.\n  - Add some more explanations to commit messages.\n  - Link to v1: https://patch.msgid.link/20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (8):\n      odb/source-packed: improve lookup when enumerating objects\n      pack-bitmap: mark object filter as `const`\n      pack-bitmap: allow aborting iteration of bitmapped objects\n      pack-bitmap: iterate object sources when opening bitmaps\n      pack-bitmap: drop `_1` suffix from functions that open bitmaps\n      pack-bitmap: introduce function to open bitmap for a single source\n      odb: introduce object filters to `odb_for_each_object()`\n      builtin/cat-file: filter objects via object database\n\n builtin/cat-file.c     |  76 +++--------------------------\n builtin/pack-objects.c |   2 +-\n builtin/rev-list.c     |   2 +-\n odb.h                  |  12 +++++\n odb/source-packed.c    |  77 ++++++++++++++++++++++++++---\n pack-bitmap.c          | 129 +++++++++++++++++++++++++++----------------------\n pack-bitmap.h          |  10 +++-\n 7 files changed, 171 insertions(+), 137 deletions(-)\n\nRange-diff versus v1:\n\n1:  7a1a92acbe ! 1:  b675967b78 odb/source-packed: improve lookup when enumerating objects\n    @@ Metadata\n      ## Commit message ##\n         odb/source-packed: improve lookup when enumerating objects\n     \n    -    When iterating through packed objects via `odb_for_each_object()` we\n    -    do so via two different mechanisms:\n    +    When iterating through objects of a packed source that have a specific\n    +    prefix we do so via two different methods:\n     \n           - When a multi-pack index is available we use that one to efficiently\n             loop through all objects.\n    @@ Commit message\n     \n           - It's subtly wrong, as it may now happen that a specific object will\n             be looked up via a different pack in case it exists multiple times.\n    +        This is unlikely to have any real-world consequences, but it's still\n    +        the wrong thing to do.\n     \n         Fix the issue by using `packed_object_info()` directly. While at it,\n         rename the `store` variable to `source`.\n2:  a6c8bd7a61 = 2:  d3f9b2f781 pack-bitmap: mark object filter as `const`\n3:  c38b06636b = 3:  825920205a pack-bitmap: allow aborting iteration of bitmapped objects\n4:  450cdd13b7 ! 4:  a33ca8fa3b pack-bitmap: iterate object sources when opening bitmaps\n    @@ pack-bitmap.c: static int load_bitmap(struct repository *r, struct bitmap_index\n     +\t\t\t\t  struct bitmap_index *bitmap_git)\n      {\n     -\tstruct packed_git *p;\n    +-\tint ret = -1;\n     +\tstruct multi_pack_index *midx = get_multi_pack_index(source);\n     +\tstruct packfile_list_entry *e;\n    - \tint ret = -1;\n    ++\tbool found = false;\n      \n     -\trepo_for_each_pack(r, p) {\n     -\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n    @@ pack-bitmap.c: static int load_bitmap(struct repository *r, struct bitmap_index\n     -\t\t\t\tbreak;\n     -\t\t}\n     +\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n    -+\t\tret = 0;\n    ++\t\tfound = true;\n     +\n     +\tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n     +\t\t/*\n     +\t\t * When tracing is enabled we want to keep looking to report\n     +\t\t * duplicates even if we have already found a bitmap.\n     +\t\t */\n    -+\t\tif (!ret && !trace2_is_enabled())\n    ++\t\tif (found && !trace2_is_enabled())\n     +\t\t\tbreak;\n     +\n    -+\t\tif (open_pack_bitmap_1(bitmap_git, e->pack))\n    -+\t\t\tcontinue;\n    -+\t\tret = 0;\n    ++\t\tif (!open_pack_bitmap_1(bitmap_git, e->pack))\n    ++\t\t\tfound = true;\n      \t}\n      \n    - \treturn ret;\n    +-\treturn ret;\n    ++\treturn found ? 0 : -1;\n      }\n      \n     -static int open_midx_bitmap(struct repository *r,\n    @@ pack-bitmap.c: static int load_bitmap(struct repository *r, struct bitmap_index\n      {\n      \tstruct odb_source *source;\n     -\tint ret = -1;\n    -+\tint found = 0;\n    ++\tbool found = false;\n      \n      \tassert(!bitmap_git->map);\n      \n    @@ pack-bitmap.c: static int load_bitmap(struct repository *r, struct bitmap_index\n     -\tint found;\n      \n     -\tassert(!bitmap_git->map);\n    -+\t\tfound |= !open_bitmap_for_source(files->packed, bitmap_git);\n    ++\t\tif (!open_bitmap_for_source(files->packed, bitmap_git))\n    ++\t\t\tfound = true;\n      \n     -\tfound = !open_midx_bitmap(r, bitmap_git);\n     -\n-:  ---------- > 5:  b890ed7163 pack-bitmap: drop `_1` suffix from functions that open bitmaps\n5:  26b1957f8b = 6:  f7e466217b pack-bitmap: introduce function to open bitmap for a single source\n6:  722727c76d ! 7:  27ecc0802f odb: introduce object filters to `odb_for_each_object()`\n    @@ Commit message\n         object filter infrastructure supports some filters that cannot be\n         answered by the object database alone.\n     \n    +    An alternative might be to limit the filters to only those that _can_ be\n    +    answered by backends. But ultimately, the filters that can be answered\n    +    efficiently by the \"packed\" backend are completely disjunct from those\n    +    that can be answered by the \"loose\" backend, and consequently the set of\n    +    filters supported by all backends would be empty. Furthermore, it would\n    +    require us to make assumptions about capabilities of future backends,\n    +    which may be able to efficiently handle more filters than current ones.\n    +    So in the end, this alternative would only limit us artificially.\n    +\n         Implement the logic for the \"packed\" source. Note that we use the new\n         function `prepare_source_bitmap_git()` to open the bitmap: as the\n         backend operates on a single object source, we must only use bitmaps\n7:  90be28e904 ! 8:  e4c6aeab0a builtin/cat-file: filter objects via object database\n    @@ Metadata\n      ## Commit message ##\n         builtin/cat-file: filter objects via object database\n     \n    +    When batching all objects, git-cat-file(1) reaches into the internals of\n    +    the object database and manually manages bitmaps to apply object\n    +    filters. This creates coupling between the command and the internals of\n    +    the respective backend.\n    +\n         Refactor git-cat-file(1) to use the new object filter option when\n         batching all objects. This significantly simplifies the logic and\n         ensures that we don't have to reach into internals of the \"files\" source\n\n---\nbase-commit: 3c8e2790f2ce15e8b5d4b4e6ced711b12649f32a\nchange-id: 20260708-pks-odb-for-each-object-filter-13286fa3523d\n\n"},{"id":"547689","messageId":"20260710-pks-odb-for-each-object-filter-v2-1-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 1/8] odb/source-packed: improve lookup when enumerating objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:48:53Z","receivedAt":"2026-07-10T08:49:07Z","isPatch":true,"body":"When iterating through objects of a packed source that have a specific\nprefix we do so via two different methods:\n\n  - When a multi-pack index is available we use that one to efficiently\n    loop through all objects.\n\n  - We then loop through all packfiles that aren't covered by a\n    multi-pack index.\n\nRegardless of which mechanism we use, we then iterate through all the\nobjects indexed by the respective data structure. Curiously though,\nwhile we use the indices for enumerating the objects, we completely\nignore it for the actual object lookup. Instead, we call into the\ngeneric `odb_source_read_object_info()` function, which will itself\nconsult the indices to figure out where the object in question even\nlives.\n\nThis has two consequences:\n\n  - It's inefficient, as we basically have to figure out the position of\n    the object a second time.\n\n  - It's subtly wrong, as it may now happen that a specific object will\n    be looked up via a different pack in case it exists multiple times.\n    This is unlikely to have any real-world consequences, but it's still\n    the wrong thing to do.\n\nFix the issue by using `packed_object_info()` directly. While at it,\nrename the `store` variable to `source`.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n odb/source-packed.c | 15 ++++++++-------\n 1 file changed, 8 insertions(+), 7 deletions(-)\n\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 0edea5356d..9cfa02b7a2 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -143,7 +143,7 @@ static bool should_exclude_pack(struct packed_git *p, enum odb_for_each_object_f\n }\n \n static int for_each_prefixed_object_in_midx(\n-\tstruct odb_source_packed *store,\n+\tstruct odb_source_packed *source,\n \tstruct multi_pack_index *m,\n \tconst struct odb_for_each_object_options *opts,\n \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n@@ -170,6 +170,7 @@ static int for_each_prefixed_object_in_midx(\n \t\t */\n \t\tfor (i = first; i < num; i++) {\n \t\t\tconst struct object_id *current = NULL;\n+\t\t\tstruct packed_git *pack;\n \t\t\tstruct object_id oid;\n \n \t\t\tcurrent = nth_midxed_object_oid(&oid, m, i);\n@@ -177,9 +178,8 @@ static int for_each_prefixed_object_in_midx(\n \t\t\tif (!match_hash(len, opts->prefix->hash, current->hash))\n \t\t\t\tbreak;\n \n-\t\t\tif (opts->flags) {\n+\t\t\tif (opts->flags || data->request) {\n \t\t\t\tuint32_t pack_id = nth_midxed_pack_int_id(m, i);\n-\t\t\t\tstruct packed_git *pack;\n \n \t\t\t\tif (prepare_midx_pack(m, pack_id)) {\n \t\t\t\t\tpack_errors = true;\n@@ -193,9 +193,9 @@ static int for_each_prefixed_object_in_midx(\n \n \t\t\tif (data->request) {\n \t\t\t\tstruct object_info oi = *data->request;\n+\t\t\t\toff_t offset = nth_midxed_offset(m, i);\n \n-\t\t\t\tret = odb_source_read_object_info(&store->base, current,\n-\t\t\t\t\t\t\t\t  &oi, 0);\n+\t\t\t\tret = packed_object_info(source, pack, offset, &oi);\n \t\t\t\tif (ret)\n \t\t\t\t\tgoto out;\n \n@@ -219,7 +219,7 @@ static int for_each_prefixed_object_in_midx(\n }\n \n static int for_each_prefixed_object_in_pack(\n-\tstruct odb_source_packed *store,\n+\tstruct odb_source_packed *source,\n \tstruct packed_git *p,\n \tconst struct odb_for_each_object_options *opts,\n \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n@@ -246,8 +246,9 @@ static int for_each_prefixed_object_in_pack(\n \n \t\tif (data->request) {\n \t\t\tstruct object_info oi = *data->request;\n+\t\t\toff_t offset = nth_packed_object_offset(p, i);\n \n-\t\t\tret = odb_source_read_object_info(&store->base, &oid, &oi, 0);\n+\t\t\tret = packed_object_info(source, p, offset, &oi);\n \t\t\tif (ret)\n \t\t\t\tgoto out;\n \n\n-- \n2.55.0.229.g6434b31f56.dirty\n\n"},{"id":"547690","messageId":"20260710-pks-odb-for-each-object-filter-v2-2-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 2/8] pack-bitmap: mark object filter as `const`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:48:54Z","receivedAt":"2026-07-10T08:49:09Z","isPatch":true,"body":"The function `for_each_bitmapped_object()` accepts an optional object\nfilter. This filter is never modified by the function, but is not\ndeclared as `const`. Fix this.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 6 +++---\n pack-bitmap.h | 2 +-\n 2 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 35774b6f0c..a47c231632 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1976,7 +1976,7 @@ static void filter_bitmap_object_type(struct bitmap_index *bitmap_git,\n static int filter_bitmap(struct bitmap_index *bitmap_git,\n \t\t\t struct object_list *tip_objects,\n \t\t\t struct bitmap *to_filter,\n-\t\t\t struct list_objects_filter_options *filter)\n+\t\t\t const struct list_objects_filter_options *filter)\n {\n \tif (!filter || filter->choice == LOFC_DISABLED)\n \t\treturn 0;\n@@ -2027,7 +2027,7 @@ static int filter_bitmap(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int can_filter_bitmap(struct list_objects_filter_options *filter)\n+static int can_filter_bitmap(const struct list_objects_filter_options *filter)\n {\n \treturn !filter_bitmap(NULL, NULL, NULL, filter);\n }\n@@ -2058,7 +2058,7 @@ static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,\n }\n \n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n-\t\t\t      struct list_objects_filter_options *filter,\n+\t\t\t      const struct list_objects_filter_options *filter,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload)\n {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 19a8655457..47935eb24e 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -96,7 +96,7 @@ struct list_objects_filter_options;\n  * not supported, `0` otherwise.\n  */\n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n-\t\t\t      struct list_objects_filter_options *filter,\n+\t\t\t      const struct list_objects_filter_options *filter,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload);\n \n\n-- \n2.55.0.229.g6434b31f56.dirty\n\n"},{"id":"547691","messageId":"20260710-pks-odb-for-each-object-filter-v2-3-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 3/8] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:48:55Z","receivedAt":"2026-07-10T08:49:13Z","isPatch":true,"body":"In a subsequent commit we'll lift iteration of bitmapped objects into\nthe \"packed\" backend and make it accessible via `odb_for_each_object()`.\nThe calling convention for that function is that the callback may return\na non-zero exit code, and if so we'll abort iteration. This is currently\nimpossible to realize though, as `for_each_bitmapped_object()` will\nignore any return value and just churn through all objects completely.\n\nThis doesn't matter to the callers of `for_each_bitmapped_object()`, as\nthere's only one of them in git-cat-file(1), and the callbacks we pass\nalways return zero. But once we move the logic into the generic\ninfrastructure it becomes a latent bug waiting to happen.\n\nRefactor the code so that the return value of the `show_reach` callback\nis not ignored anymore. Instead, returning a non-zero value will cause\nus to abort iteration in both `show_objects_for_type()` and in\n`for_each_bitmapped_object()`.\n\nNote though that there's a second user of `show_objects_for_type()` with\n`traverse_bitmap_commit_list()`, and that function does indeed invoke\ncallbacks that may return non-zero. This non-zero return value never had\nany effect at all though, and the callbacks that return non-zero values\nare only ever invoked via `traverse_bitmap_commit_list()`. Consequently,\nwe adapt them to always return 0.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/pack-objects.c |  2 +-\n builtin/rev-list.c     |  2 +-\n pack-bitmap.c          | 31 +++++++++++++++++++++----------\n pack-bitmap.h          |  3 ++-\n 4 files changed, 25 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex ea5eab4cf8..8ff92c5272 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1909,7 +1909,7 @@ static int add_object_entry_from_bitmap(const struct object_id *oid,\n \t\treturn 0;\n \n \tcreate_object_entry(oid, type, name_hash, 0, 0, pack, offset);\n-\treturn 1;\n+\treturn 0;\n }\n \n struct pbase_tree_cache {\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 8f63003709..02818b81c6 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -486,7 +486,7 @@ static int show_object_fast(\n \tvoid *payload UNUSED)\n {\n \tfprintf(stdout, \"%s\\n\", oid_to_hex(oid));\n-\treturn 1;\n+\treturn 0;\n }\n \n static void print_disk_usage(off_t size)\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex a47c231632..eda38a5433 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1695,7 +1695,7 @@ static void init_type_iterator(struct ewah_or_iterator *it,\n \t}\n }\n \n-static void show_objects_for_type(\n+static int show_objects_for_type(\n \tstruct bitmap_index *bitmap_git,\n \tstruct bitmap *objects,\n \tenum object_type object_type,\n@@ -1704,6 +1704,7 @@ static void show_objects_for_type(\n {\n \tsize_t i = 0;\n \tuint32_t offset;\n+\tint ret;\n \n \tstruct ewah_or_iterator it;\n \teword_t filter;\n@@ -1749,11 +1750,17 @@ static void show_objects_for_type(\n \n \t\t\thash = bitmap_name_hash(bitmap_git, index_pos);\n \n-\t\t\tshow_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n+\t\t\tret = show_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n+\t\t\tif (ret)\n+\t\t\t\tgoto out;\n \t\t}\n \t}\n \n+\tret = 0;\n+\n+out:\n \tewah_or_iterator_release(&it);\n+\treturn ret;\n }\n \n static int in_bitmapped_pack(struct bitmap_index *bitmap_git,\n@@ -2062,6 +2069,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload)\n {\n+\tconst enum object_type types[] = {\n+\t\tOBJ_COMMIT,\n+\t\tOBJ_TREE,\n+\t\tOBJ_BLOB,\n+\t\tOBJ_TAG,\n+\t};\n \tstruct bitmap *filtered_bitmap = NULL;\n \tuint32_t objects_nr;\n \tsize_t full_word_count;\n@@ -2086,14 +2099,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\tgoto out;\n \t}\n \n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_COMMIT, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_TREE, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_BLOB, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_TAG, show_reach, payload);\n+\tfor (size_t i = 0; i < ARRAY_SIZE(types); i++) {\n+\t\tret = show_objects_for_type(bitmap_git, filtered_bitmap,\n+\t\t\t\t\t    types[i], show_reach, payload);\n+\t\tif (ret)\n+\t\t\tgoto out;\n+\t}\n \n \tret = 0;\n out:\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 47935eb24e..ae8dc491ac 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -93,7 +93,8 @@ struct list_objects_filter_options;\n /*\n  * Filter bitmapped objects and iterate through all resulting objects,\n  * executing `show_reach` for each of them. Returns `-1` in case the filter is\n- * not supported, `0` otherwise.\n+ * not supported, `0` otherwise. Aborts iteration and bubbles up the return\n+ * value in case `show_reach()` returns non-zero.\n  */\n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\t\t      const struct list_objects_filter_options *filter,\n\n-- \n2.55.0.229.g6434b31f56.dirty\n\n"},{"id":"547692","messageId":"20260710-pks-odb-for-each-object-filter-v2-4-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 4/8] pack-bitmap: iterate object sources when opening bitmaps","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:48:56Z","receivedAt":"2026-07-10T08:49:16Z","isPatch":true,"body":"When opening a bitmap for a repository we perform two steps:\n\n  - We first look for a multi-pack index bitmap in any of the object\n    sources connected to the repository.\n\n  - We then look for a packfile bitmap in any of the packfiles of any of\n    the object sources.\n\nBoth of these steps thus iterate through object sources themselves, one\nvia `odb_prepare_alternates()` and one via `repo_for_each_pack()`. This\nlayout makes it hard to introduce a way to open the bitmap of one\nspecific object source, which is functionality that we'll require in a\nsubsequent commit.\n\nReverse the loop so that we instead loop through all sources in the\nouter loop, and then for each source we try to load its bitmap via\neither the multi-pack index or via a packfile.\n\nNote that this changes the precedence of bitmaps in one specific edge\ncase: when an earlier object source only has a packfile bitmap, but a\nlater source has a multi-pack index bitmap, we now pick the packfile\nbitmap of the earlier source. Previously, a multi-pack index bitmap from\nany source would have taken precedence over all packfile bitmaps. Given\nthat object sources are ordered such that the local source comes first,\nthis arguably is an improvement, as we now prefer local bitmaps over\nbitmaps in alternates. Furthermore, we already warn about repositories\nthat have multiple bitmaps, so this setup is broken and thus arguably\nnot worth worrying about too much.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 69 +++++++++++++++++++++++++++--------------------------------\n 1 file changed, 31 insertions(+), 38 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex eda38a5433..e32795a595 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -680,60 +680,53 @@ static int load_bitmap(struct repository *r, struct bitmap_index *bitmap_git,\n \treturn 0;\n }\n \n-static int open_pack_bitmap(struct repository *r,\n-\t\t\t    struct bitmap_index *bitmap_git)\n+static int open_bitmap_for_source(struct odb_source_packed *source,\n+\t\t\t\t  struct bitmap_index *bitmap_git)\n {\n-\tstruct packed_git *p;\n-\tint ret = -1;\n+\tstruct multi_pack_index *midx = get_multi_pack_index(source);\n+\tstruct packfile_list_entry *e;\n+\tbool found = false;\n \n-\trepo_for_each_pack(r, p) {\n-\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n-\t\t\tret = 0;\n-\t\t\t/*\n-\t\t\t * The only reason to keep looking is to report\n-\t\t\t * duplicates.\n-\t\t\t */\n-\t\t\tif (!trace2_is_enabled())\n-\t\t\t\tbreak;\n-\t\t}\n+\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n+\t\tfound = true;\n+\n+\tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n+\t\t/*\n+\t\t * When tracing is enabled we want to keep looking to report\n+\t\t * duplicates even if we have already found a bitmap.\n+\t\t */\n+\t\tif (found && !trace2_is_enabled())\n+\t\t\tbreak;\n+\n+\t\tif (!open_pack_bitmap_1(bitmap_git, e->pack))\n+\t\t\tfound = true;\n \t}\n \n-\treturn ret;\n+\treturn found ? 0 : -1;\n }\n \n-static int open_midx_bitmap(struct repository *r,\n-\t\t\t    struct bitmap_index *bitmap_git)\n+static int open_bitmap(struct repository *r,\n+\t\t       struct bitmap_index *bitmap_git)\n {\n \tstruct odb_source *source;\n-\tint ret = -1;\n+\tbool found = false;\n \n \tassert(!bitmap_git->map);\n \n \todb_prepare_alternates(r->objects);\n \tfor (source = r->objects->sources; source; source = source->next) {\n \t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\tstruct multi_pack_index *midx = get_multi_pack_index(files->packed);\n-\t\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\tret = 0;\n-\t}\n-\treturn ret;\n-}\n-\n-static int open_bitmap(struct repository *r,\n-\t\t       struct bitmap_index *bitmap_git)\n-{\n-\tint found;\n \n-\tassert(!bitmap_git->map);\n+\t\tif (!open_bitmap_for_source(files->packed, bitmap_git))\n+\t\t\tfound = true;\n \n-\tfound = !open_midx_bitmap(r, bitmap_git);\n-\n-\t/*\n-\t * these will all be skipped if we opened a midx bitmap; but run it\n-\t * anyway if tracing is enabled to report the duplicates\n-\t */\n-\tif (!found || trace2_is_enabled())\n-\t\tfound |= !open_pack_bitmap(r, bitmap_git);\n+\t\t/*\n+\t\t * The only reason to keep looking after having found a bitmap\n+\t\t * is to report duplicates.\n+\t\t */\n+\t\tif (found && !trace2_is_enabled())\n+\t\t\tbreak;\n+\t}\n \n \treturn found ? 0 : -1;\n }\n\n-- \n2.55.0.229.g6434b31f56.dirty\n\n"},{"id":"547693","messageId":"20260710-pks-odb-for-each-object-filter-v2-5-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 5/8] pack-bitmap: drop `_1` suffix from functions that open bitmaps","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:48:57Z","receivedAt":"2026-07-10T08:49:19Z","isPatch":true,"body":"In the preceding commit we've refactored how we open bitmaps. As part of\nthe refactoring we have consolidated `open_pack_bitmap()` as well as\n`open_midx_bitmap()` into `open_bitmap_for_source()`. Consequently, we\nonly have their `open_pack_bitmap_1()` and `open_midx_bitmap_1()`\nvariants left over, where the `_1` suffix doesn't really make much sense\nanymore.\n\nDrop the suffix.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex e32795a595..72c8ae3228 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -460,8 +460,8 @@ char *pack_bitmap_filename(struct packed_git *p)\n \treturn xstrfmt(\"%.*s.bitmap\", (int)len, p->pack_name);\n }\n \n-static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n-\t\t\t      struct multi_pack_index *midx)\n+static int open_midx_bitmap(struct bitmap_index *bitmap_git,\n+\t\t\t    struct multi_pack_index *midx)\n {\n \tstruct stat st;\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n@@ -539,7 +539,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n+static int open_pack_bitmap(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n {\n \tint fd;\n \tstruct stat st;\n@@ -603,7 +603,7 @@ static int load_reverse_index(struct repository *r, struct bitmap_index *bitmap_\n \n \t\t/*\n \t\t * The multi-pack-index's .rev file is already loaded via\n-\t\t * open_pack_bitmap_1().\n+\t\t * open_pack_bitmap().\n \t\t *\n \t\t * But we still need to open the individual pack .rev files,\n \t\t * since we will need to make use of them in pack-objects.\n@@ -687,7 +687,7 @@ static int open_bitmap_for_source(struct odb_source_packed *source,\n \tstruct packfile_list_entry *e;\n \tbool found = false;\n \n-\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n+\tif (midx && !open_midx_bitmap(bitmap_git, midx))\n \t\tfound = true;\n \n \tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n@@ -698,7 +698,7 @@ static int open_bitmap_for_source(struct odb_source_packed *source,\n \t\tif (found && !trace2_is_enabled())\n \t\t\tbreak;\n \n-\t\tif (!open_pack_bitmap_1(bitmap_git, e->pack))\n+\t\tif (!open_pack_bitmap(bitmap_git, e->pack))\n \t\t\tfound = true;\n \t}\n \n@@ -746,7 +746,7 @@ struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx)\n {\n \tstruct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git));\n \n-\tif (!open_midx_bitmap_1(bitmap_git, midx))\n+\tif (!open_midx_bitmap(bitmap_git, midx))\n \t\treturn bitmap_git;\n \n \tfree_bitmap_index(bitmap_git);\n\n-- \n2.55.0.229.g6434b31f56.dirty\n\n"},{"id":"547694","messageId":"20260710-pks-odb-for-each-object-filter-v2-6-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 6/8] pack-bitmap: introduce function to open bitmap for a single source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:48:58Z","receivedAt":"2026-07-10T08:49:22Z","isPatch":true,"body":"The function `prepare_bitmap_git()` opens the first bitmap it can find\nin any of the object sources connected to the repository. In a\nsubsequent commit, the \"packed\" object database backend will learn to\nuse bitmaps to answer object filters when enumerating objects. That\nbackend operates on a single object source though, so using a bitmap\nthat potentially belongs to a different source would be wrong:\n\n  - The source would yield objects that are not part of the source\n    itself.\n\n  - The object source info would be attributed to the wrong source.\n\n  - With multiple sources, each source would enumerate the same bitmap\n    another time.\n\nIntroduce a new function `prepare_source_bitmap_git()` that only opens\nbitmaps belonging to the given object source.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 12 ++++++++++++\n pack-bitmap.h |  2 ++\n 2 files changed, 14 insertions(+)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 72c8ae3228..09ba15d26b 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -753,6 +753,18 @@ struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx)\n \treturn NULL;\n }\n \n+struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source)\n+{\n+\tstruct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git));\n+\n+\tif (!open_bitmap_for_source(source, bitmap_git) &&\n+\t    !load_bitmap(source->base.odb->repo, bitmap_git, 0))\n+\t\treturn bitmap_git;\n+\n+\tfree_bitmap_index(bitmap_git);\n+\treturn NULL;\n+}\n+\n int bitmap_index_contains_pack(struct bitmap_index *bitmap, struct packed_git *pack)\n {\n \tfor (; bitmap; bitmap = bitmap->base) {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex ae8dc491ac..9f20fb6e56 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -9,6 +9,7 @@\n #include \"string-list.h\"\n \n struct commit;\n+struct odb_source_packed;\n struct repository;\n struct rev_info;\n \n@@ -68,6 +69,7 @@ struct bitmapped_pack {\n \n struct bitmap_index *prepare_bitmap_git(struct repository *r);\n struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx);\n+struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source);\n \n /*\n  * Given a bitmap index, determine whether it contains the pack either directly\n\n-- \n2.55.0.229.g6434b31f56.dirty\n\n"},{"id":"547695","messageId":"20260710-pks-odb-for-each-object-filter-v2-7-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 7/8] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:48:59Z","receivedAt":"2026-07-10T08:49:25Z","isPatch":true,"body":"The function `for_each_bitmapped_object()` can be used to iterate\nthrough all objects covered by a bitmap. The benefit of this function is\nthat it allows the caller to efficiently handle some object filters. For\nexample, this can be used to filter out objects of a specific type with\nsome simple bitmap operations. But callers are currently required to\nmanually wire up the use of bitmaps though, and to do so they have to\nreach into internals of a given object database source.\n\nIntroduce a new `struct odb_for_each_object_options::filter` field so\nthat the interface becomes generic. When set, then a backend may\noptionally use the filter to skip some objects that it would have\notherwise yielded.\n\nNote that the respective backends are free to ignore this field if they\ncannot meaningfully optimize for a given filter, and consequently\ncallers need to verify whether they actually want the returned objects.\nWhile annoying, we cannot easily lift this restriction anyway as the\nobject filter infrastructure supports some filters that cannot be\nanswered by the object database alone.\n\nAn alternative might be to limit the filters to only those that _can_ be\nanswered by backends. But ultimately, the filters that can be answered\nefficiently by the \"packed\" backend are completely disjunct from those\nthat can be answered by the \"loose\" backend, and consequently the set of\nfilters supported by all backends would be empty. Furthermore, it would\nrequire us to make assumptions about capabilities of future backends,\nwhich may be able to efficiently handle more filters than current ones.\nSo in the end, this alternative would only limit us artificially.\n\nImplement the logic for the \"packed\" source. Note that we use the new\nfunction `prepare_source_bitmap_git()` to open the bitmap: as the\nbackend operates on a single object source, we must only use bitmaps\nthat belong to that specific source. Otherwise we might yield objects\nthat are not part of the source at all, and with multiple sources we\nwould enumerate the same bitmap once per source.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n odb.h               | 12 +++++++++++\n odb/source-packed.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n pack-bitmap.c       |  3 +--\n pack-bitmap.h       |  3 +++\n 4 files changed, 78 insertions(+), 2 deletions(-)\n\ndiff --git a/odb.h b/odb.h\nindex a1e222f605..67d0b34942 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -8,6 +8,7 @@\n #include \"thread-utils.h\"\n \n struct cached_object_entry;\n+struct list_objects_filter_options;\n struct odb_source_inmemory;\n struct packed_git;\n struct repository;\n@@ -490,6 +491,17 @@ struct odb_for_each_object_options {\n \t */\n \tconst struct object_id *prefix;\n \tsize_t prefix_hex_len;\n+\n+\t/*\n+\t * Optional object filter that allows backends to skip yielding\n+\t * objects that are excluded by the filter as an optimization. The\n+\t * filter is a best-effort hint: backends may use it to skip\n+\t * excluded objects (e.g. by consulting a reachability bitmap), but\n+\t * are also free to ignore it entirely and yield every object. As a\n+\t * consequence, callers must re-apply the filter on yielded objects\n+\t * if they require strict filtering semantics.\n+\t */\n+\tconst struct list_objects_filter_options *filter;\n };\n \n /*\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 9cfa02b7a2..4777395053 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -3,11 +3,13 @@\n #include \"chdir-notify.h\"\n #include \"dir.h\"\n #include \"git-zlib.h\"\n+#include \"list-objects-filter-options.h\"\n #include \"mergesort.h\"\n #include \"midx.h\"\n #include \"odb/source-packed.h\"\n #include \"odb/streaming.h\"\n #include \"packfile.h\"\n+#include \"pack-bitmap.h\"\n \n static int find_pack_entry(struct odb_source_packed *store,\n \t\t\t   const struct object_id *oid,\n@@ -315,6 +317,37 @@ static int odb_source_packed_for_each_prefixed_object(\n \treturn ret;\n }\n \n+struct bitmapped_for_each_object_data {\n+\tstruct odb_source_packed *packed;\n+\tconst struct object_info *request;\n+\tconst struct odb_for_each_object_options *opts;\n+\todb_for_each_object_cb cb;\n+\tvoid *cb_data;\n+};\n+\n+static int bitmapped_for_each_object(const struct object_id *oid,\n+\t\t\t\t     enum object_type type UNUSED,\n+\t\t\t\t     int flags UNUSED,\n+\t\t\t\t     uint32_t hash UNUSED,\n+\t\t\t\t     struct packed_git *pack,\n+\t\t\t\t     off_t offset,\n+\t\t\t\t     void *cb_data)\n+{\n+\tstruct bitmapped_for_each_object_data *data = cb_data;\n+\n+\tif (should_exclude_pack(pack, data->opts->flags))\n+\t\treturn 0;\n+\n+\tif (data->request) {\n+\t\tstruct object_info oi = *data->request;\n+\t\tif (packed_object_info(data->packed, pack, offset, &oi) < 0)\n+\t\t\treturn -1;\n+\t\treturn data->cb(oid, &oi, data->cb_data);\n+\t}\n+\n+\treturn data->cb(oid, NULL, data->cb_data);\n+}\n+\n static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\t\t\t\t     const struct object_info *request,\n \t\t\t\t\t     odb_for_each_object_cb cb,\n@@ -328,12 +361,33 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\t.cb = cb,\n \t\t.cb_data = cb_data,\n \t};\n+\tstruct bitmap_index *bitmap = NULL;\n \tstruct packfile_list_entry *e;\n \tint pack_errors = 0, ret;\n \n \tif (opts->prefix)\n \t\treturn odb_source_packed_for_each_prefixed_object(packed, opts, &data);\n \n+\tif (opts->filter &&\n+\t    opts->filter->choice != LOFC_DISABLED &&\n+\t    can_filter_bitmap(opts->filter))\n+\t\tbitmap = prepare_bitmap_git_for_source(packed);\n+\tif (bitmap) {\n+\t\tstruct bitmapped_for_each_object_data bitmap_data = {\n+\t\t\t.packed = packed,\n+\t\t\t.request = request,\n+\t\t\t.opts = opts,\n+\t\t\t.cb = cb,\n+\t\t\t.cb_data = cb_data,\n+\t\t};\n+\n+\t\tret = for_each_bitmapped_object(bitmap, opts->filter,\n+\t\t\t\t\t\tbitmapped_for_each_object,\n+\t\t\t\t\t\t&bitmap_data);\n+\t\tif (ret)\n+\t\t\tgoto out;\n+\t}\n+\n \tpacked->skip_mru_updates = true;\n \n \tfor (e = packfile_store_get_packs(packed); e; e = e->next) {\n@@ -342,6 +396,13 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\tif (should_exclude_pack(p, opts->flags))\n \t\t\tcontinue;\n \n+\t\t/*\n+\t\t * Objects covered by the bitmap have already been yielded\n+\t\t * above; skip them here to avoid duplicates.\n+\t\t */\n+\t\tif (bitmap && bitmap_index_contains_pack(bitmap, p))\n+\t\t\tcontinue;\n+\n \t\tif (open_pack_index(p)) {\n \t\t\tpack_errors = 1;\n \t\t\tcontinue;\n@@ -357,6 +418,7 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \n out:\n \tpacked->skip_mru_updates = false;\n+\tfree_bitmap_index(bitmap);\n \n \tif (!ret && pack_errors)\n \t\tret = -1;\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 09ba15d26b..f55a0859ea 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -2039,12 +2039,11 @@ static int filter_bitmap(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int can_filter_bitmap(const struct list_objects_filter_options *filter)\n+bool can_filter_bitmap(const struct list_objects_filter_options *filter)\n {\n \treturn !filter_bitmap(NULL, NULL, NULL, filter);\n }\n \n-\n static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,\n \t\t\t\t\t      struct bitmap *result)\n {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 9f20fb6e56..1385027c1f 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -92,6 +92,9 @@ int test_bitmap_pseudo_merge_objects(struct repository *r, uint32_t n);\n \n struct list_objects_filter_options;\n \n+/* Check whether the filter can be computed via the bitmap. */\n+bool can_filter_bitmap(const struct list_objects_filter_options *filter);\n+\n /*\n  * Filter bitmapped objects and iterate through all resulting objects,\n  * executing `show_reach` for each of them. Returns `-1` in case the filter is\n\n-- \n2.55.0.229.g6434b31f56.dirty\n\n"},{"id":"547696","messageId":"20260710-pks-odb-for-each-object-filter-v2-8-3710a9cc165a@pks.im","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 8/8] builtin/cat-file: filter objects via object database","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-10T08:49:00Z","receivedAt":"2026-07-10T08:49:28Z","isPatch":true,"body":"When batching all objects, git-cat-file(1) reaches into the internals of\nthe object database and manually manages bitmaps to apply object\nfilters. This creates coupling between the command and the internals of\nthe respective backend.\n\nRefactor git-cat-file(1) to use the new object filter option when\nbatching all objects. This significantly simplifies the logic and\nensures that we don't have to reach into internals of the \"files\" source\nanymore.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/cat-file.c | 76 +++++-------------------------------------------------\n 1 file changed, 7 insertions(+), 69 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex b4b99a73da..1458dd76d6 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -20,7 +20,6 @@\n #include \"userdiff.h\"\n #include \"oid-array.h\"\n #include \"packfile.h\"\n-#include \"pack-bitmap.h\"\n #include \"object-file.h\"\n #include \"object-name.h\"\n #include \"odb.h\"\n@@ -844,28 +843,6 @@ static int batch_one_object_oi(const struct object_id *oid,\n \treturn payload->callback(oid, NULL, 0, payload->payload);\n }\n \n-static int batch_one_object_packed(const struct object_id *oid,\n-\t\t\t\t   struct packed_git *pack,\n-\t\t\t\t   uint32_t pos,\n-\t\t\t\t   void *_payload)\n-{\n-\tstruct for_each_object_payload *payload = _payload;\n-\treturn payload->callback(oid, pack, nth_packed_object_offset(pack, pos),\n-\t\t\t\t payload->payload);\n-}\n-\n-static int batch_one_object_bitmapped(const struct object_id *oid,\n-\t\t\t\t      enum object_type type UNUSED,\n-\t\t\t\t      int flags UNUSED,\n-\t\t\t\t      uint32_t hash UNUSED,\n-\t\t\t\t      struct packed_git *pack,\n-\t\t\t\t      off_t offset,\n-\t\t\t\t      void *_payload)\n-{\n-\tstruct for_each_object_payload *payload = _payload;\n-\treturn payload->callback(oid, pack, offset, payload->payload);\n-}\n-\n static void batch_each_object(struct batch_options *opt,\n \t\t\t      for_each_object_fn callback,\n \t\t\t      unsigned flags,\n@@ -875,56 +852,17 @@ static void batch_each_object(struct batch_options *opt,\n \t\t.callback = callback,\n \t\t.payload = _payload,\n \t};\n+\tstruct odb_source_info source_info;\n+\tstruct object_info oi = {\n+\t\t.source_infop = &source_info,\n+\t};\n \tstruct odb_for_each_object_options opts = {\n \t\t.flags = flags,\n+\t\t.filter = &opt->objects_filter,\n \t};\n-\tstruct bitmap_index *bitmap = NULL;\n-\tstruct odb_source *source;\n-\n-\t/*\n-\t * TODO: we still need to tap into implementation details of the object\n-\t * database sources. Ideally, we should extend `odb_for_each_object()`\n-\t * to handle object filters itself so that we can move the filtering\n-\t * logic into the individual sources.\n-\t */\n-\todb_prepare_alternates(the_repository->objects);\n-\tfor (source = the_repository->objects->sources; source; source = source->next) {\n-\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\tint ret = odb_source_for_each_object(&files->loose->base, NULL, batch_one_object_oi,\n-\t\t\t\t\t\t     &payload, &opts);\n-\t\tif (ret)\n-\t\t\tbreak;\n-\t}\n-\n-\tif (opt->objects_filter.choice != LOFC_DISABLED &&\n-\t    (bitmap = prepare_bitmap_git(the_repository)) &&\n-\t    !for_each_bitmapped_object(bitmap, &opt->objects_filter,\n-\t\t\t\t       batch_one_object_bitmapped, &payload)) {\n-\t\tstruct packed_git *pack;\n-\n-\t\trepo_for_each_pack(the_repository, pack) {\n-\t\t\tif (bitmap_index_contains_pack(bitmap, pack) ||\n-\t\t\t    open_pack_index(pack))\n-\t\t\t\tcontinue;\n-\t\t\tfor_each_object_in_pack(pack, batch_one_object_packed,\n-\t\t\t\t\t\t&payload, flags);\n-\t\t}\n-\t} else {\n-\t\tstruct odb_source_info source_info;\n-\t\tstruct object_info oi = {\n-\t\t\t.source_infop = &source_info,\n-\t\t};\n-\n-\t\tfor (source = the_repository->objects->sources; source; source = source->next) {\n-\t\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\t\tint ret = odb_source_for_each_object(&files->packed->base, &oi,\n-\t\t\t\t\t\t\t     batch_one_object_oi, &payload, &opts);\n-\t\t\tif (ret)\n-\t\t\t\tbreak;\n-\t\t}\n-\t}\n \n-\tfree_bitmap_index(bitmap);\n+\todb_for_each_object_ext(the_repository->objects, &oi,\n+\t\t\t\tbatch_one_object_oi, &payload, &opts);\n }\n \n static int batch_objects(struct batch_options *opt)\n\n-- \n2.55.0.229.g6434b31f56.dirty\n\n"},{"id":"547802","messageId":"alFxRvkfNgJRCQTB@com-79390","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-1-3710a9cc165a@pks.im","subject":"Re: [PATCH v2 1/8] odb/source-packed: improve lookup when enumerating objects","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-10T22:25:10Z","receivedAt":"2026-07-10T22:25:14Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 10:48:53AM +0200, Patrick Steinhardt wrote:\n> Fix the issue by using `packed_object_info()` directly.\n\nWhat you wrote here makes sense to me insofar as I understand the\npluggable ODB code.\n\nHowever, I am confused by the way this function is written in general.\nWe use `bsearch_one_midx()` to locate the first possible MIDX position\nin which an object matching the given prefix may exist, which is\nsensible. However, we go from that position up to \"num\", where \"num\" is\nthe total number of objects in the MIDX!\n\nFunctionally this is not incorrect as we will happily discard objects\nthat do not match the prefix. But it causes us to waste CPU cycles\nrepeatedly calling `match_hash()` (at least for the first byte of the\nprefix) for objects that we know will match.\n\nHow often do we call this function with a prefix longer than a\nsingle byte? I have no idea, but I would suspect that it makes up the\nmajority of calls. If we read the OID fanout chunk, we could narrow the\nrange that we enumerate through, and only compare the second byte\nonwards of the given prefix, if one exists. In the single-byte prefix\ncase, this means that we shouldn't have to do any memory comparisons at\nall.\n\n> While at it, rename the `store` variable to `source`.\n\nUnrelated, but please keep these to a minimum, as they make the patch\nmore difficult to read than is necessary.\n\nThanks,\nTaylor\n"},{"id":"547803","messageId":"alFxdsQz9zRMrScO@com-79390","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-2-3710a9cc165a@pks.im","subject":"Re: [PATCH v2 2/8] pack-bitmap: mark object filter as `const`","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-10T22:25:58Z","receivedAt":"2026-07-10T22:26:02Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 10:48:54AM +0200, Patrick Steinhardt wrote:\n> The function `for_each_bitmapped_object()` accepts an optional object\n> filter. This filter is never modified by the function, but is not\n> declared as `const`. Fix this.\n\nMakes sense. \"Fix\" this seems to imply that the existing behavior was\nbroken or otherwise incorrect, but I think this is fine.\n\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  pack-bitmap.c | 6 +++---\n>  pack-bitmap.h | 2 +-\n>  2 files changed, 4 insertions(+), 4 deletions(-)\n\nThanks,\nTaylor\n"},{"id":"547805","messageId":"alFzja98avOoKjQE@com-79390","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-3-3710a9cc165a@pks.im","subject":"Re: [PATCH v2 3/8] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-10T22:34:53Z","receivedAt":"2026-07-10T22:34:57Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 10:48:55AM +0200, Patrick Steinhardt wrote:\n> In a subsequent commit we'll lift iteration of bitmapped objects into\n> the \"packed\" backend and make it accessible via `odb_for_each_object()`.\n> The calling convention for that function is that the callback may return\n> a non-zero exit code, and if so we'll abort iteration. This is currently\n> impossible to realize though, as `for_each_bitmapped_object()` will\n> ignore any return value and just churn through all objects completely.\n>\n> This doesn't matter to the callers of `for_each_bitmapped_object()`, as\n> there's only one of them in git-cat-file(1), and the callbacks we pass\n> always return zero. But once we move the logic into the generic\n> infrastructure it becomes a latent bug waiting to happen.\n\nMakes sense.\n\n> diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> index ea5eab4cf8..8ff92c5272 100644\n> --- a/builtin/pack-objects.c\n> +++ b/builtin/pack-objects.c\n> @@ -1909,7 +1909,7 @@ static int add_object_entry_from_bitmap(const struct object_id *oid,\n>  \t\treturn 0;\n>\n>  \tcreate_object_entry(oid, type, name_hash, 0, 0, pack, offset);\n> -\treturn 1;\n> +\treturn 0;\n>  }\n\nI was initially rather surprised to read this diff. I suspected that\nthis was a \"we used to return non-zero to indicate success but now\nreturn zero to match the project conventions\", but was stumped by the\nunchanged \"return 0\" in the context above.\n\nBut I suppose that is demonstrating the thing that you're trying to fix\nhere, which is that the caller doesn't actually care what is returned\nfrom the callback, so the change here (and analogous ones below) make\nsense to me.\n\n> -static void show_objects_for_type(\n> +static int show_objects_for_type(\n>  \tstruct bitmap_index *bitmap_git,\n>  \tstruct bitmap *objects,\n>  \tenum object_type object_type,\n> @@ -1704,6 +1704,7 @@ static void show_objects_for_type(\n>  {\n>  \tsize_t i = 0;\n>  \tuint32_t offset;\n> +\tint ret;\n\nThis has a broader scope than is strictly necessary, but I think that is\nOK.\n\n>  static int in_bitmapped_pack(struct bitmap_index *bitmap_git,\n> @@ -2062,6 +2069,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n>  \t\t\t      show_reachable_fn show_reach,\n>  \t\t\t      void *payload)\n>  {\n> +\tconst enum object_type types[] = {\n> +\t\tOBJ_COMMIT,\n> +\t\tOBJ_TREE,\n> +\t\tOBJ_BLOB,\n> +\t\tOBJ_TAG,\n> +\t};\n>  \tstruct bitmap *filtered_bitmap = NULL;\n>  \tuint32_t objects_nr;\n>  \tsize_t full_word_count;\n> @@ -2086,14 +2099,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n>  \t\tgoto out;\n>  \t}\n>\n> -\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n> -\t\t\t      OBJ_COMMIT, show_reach, payload);\n> -\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n> -\t\t\t      OBJ_TREE, show_reach, payload);\n> -\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n> -\t\t\t      OBJ_BLOB, show_reach, payload);\n> -\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n> -\t\t\t      OBJ_TAG, show_reach, payload);\n> +\tfor (size_t i = 0; i < ARRAY_SIZE(types); i++) {\n> +\t\tret = show_objects_for_type(bitmap_git, filtered_bitmap,\n> +\t\t\t\t\t    types[i], show_reach, payload);\n> +\t\tif (ret)\n> +\t\t\tgoto out;\n> +\t}\n\nOK. So now we call this function in a loop instead of the unrolled\nversion, presumably because we want to propagate a failure from any one\nof these before falling through to the remaining object types.\n\nThat makes sense, and I think the clean-up is well justified here.\n\nHowever, the remaining `show_objects_for_type()` callers from within\n`traverse_bitmap_commit_list()` do *not* bother to inspect the return\nvalue, despite taking in an arbitrary 'show_reachable_fn', which itself\nmay return a non-zero value.\n\nI guess this must be effectively OK in practice with respect to the\nexisting code for the same reason you indicate in the commit message\nabove, but we should change this function to *also* propagate non-zero\nreturn values to eliminate the foot-gun completely.\n\nThanks,\nTaylor\n"},{"id":"547806","messageId":"alF04ypsYm01y1Cg@com-79390","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-4-3710a9cc165a@pks.im","subject":"Re: [PATCH v2 4/8] pack-bitmap: iterate object sources when opening bitmaps","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-10T22:40:35Z","receivedAt":"2026-07-10T22:40:40Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 10:48:56AM +0200, Patrick Steinhardt wrote:\n> When opening a bitmap for a repository we perform two steps:\n>\n>   - We first look for a multi-pack index bitmap in any of the object\n>     sources connected to the repository.\n>\n>   - We then look for a packfile bitmap in any of the packfiles of any of\n>     the object sources.\n>\n> Both of these steps thus iterate through object sources themselves, one\n> via `odb_prepare_alternates()` and one via `repo_for_each_pack()`. This\n> layout makes it hard to introduce a way to open the bitmap of one\n> specific object source, which is functionality that we'll require in a\n> subsequent commit.\n>\n> Reverse the loop so that we instead loop through all sources in the\n> outer loop, and then for each source we try to load its bitmap via\n> either the multi-pack index or via a packfile.\n\nThis makes sense. An individual object store should be considered to\nhave a bitmap in the abstract sense if it provides either a multi-pack\nbitmap (or an incremental multi-pack bitmap ), or a single-pack\nbitmap.\n\n> Note that this changes the precedence of bitmaps in one specific edge\n> case: when an earlier object source only has a packfile bitmap, but a\n> later source has a multi-pack index bitmap, we now pick the packfile\n> bitmap of the earlier source. Previously, a multi-pack index bitmap from\n> any source would have taken precedence over all packfile bitmaps. Given\n> that object sources are ordered such that the local source comes first,\n> this arguably is an improvement, as we now prefer local bitmaps over\n> bitmaps in alternates. Furthermore, we already warn about repositories\n> that have multiple bitmaps, so this setup is broken and thus arguably\n> not worth worrying about too much.\n\nYeah, I think the existing behavior should be considered broken, so I\nthink that this behavior change is a positive one.\n\nThanks,\nTaylor\n"},{"id":"547807","messageId":"alF1E3rKH2lsax84@com-79390","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-5-3710a9cc165a@pks.im","subject":"Re: [PATCH v2 5/8] pack-bitmap: drop `_1` suffix from functions that open bitmaps","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-10T22:41:23Z","receivedAt":"2026-07-10T22:41:28Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 10:48:57AM +0200, Patrick Steinhardt wrote:\n> In the preceding commit we've refactored how we open bitmaps. As part of\n> the refactoring we have consolidated `open_pack_bitmap()` as well as\n> `open_midx_bitmap()` into `open_bitmap_for_source()`. Consequently, we\n> only have their `open_pack_bitmap_1()` and `open_midx_bitmap_1()`\n> variants left over, where the `_1` suffix doesn't really make much sense\n> anymore.\n>\n> Drop the suffix.\n\nMakes sense. Thanks for keeping this in a separate commit.\n\nThanks,\nTaylor\n"},{"id":"547808","messageId":"alF1bmRwwutpLQ7W@com-79390","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-7-3710a9cc165a@pks.im","subject":"Re: [PATCH v2 7/8] odb: introduce object filters to `odb_for_each_object()`","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-10T22:42:54Z","receivedAt":"2026-07-10T22:42:58Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 10:48:59AM +0200, Patrick Steinhardt wrote:\n> ---\n>  odb.h               | 12 +++++++++++\n>  odb/source-packed.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n>  pack-bitmap.c       |  3 +--\n>  pack-bitmap.h       |  3 +++\n>  4 files changed, 78 insertions(+), 2 deletions(-)\n\nThis all looks about as expected to me. As mentioned earlier in this\nthread, I am not as familiar with the pluggable ODB code as I'd like to\nbe, but the patch looks plausibly correct to me.\n\nThanks,\nTaylor\n"},{"id":"547813","messageId":"20260711074723.GB1457061@coredump.intra.peff.net","threadId":"65959","inReplyTo":"alCafO91ZtFdikPg@pks.im","subject":"Re: [PATCH 3/7] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-07-11T07:47:23Z","receivedAt":"2026-07-11T07:47:24Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 09:08:44AM +0200, Patrick Steinhardt wrote:\n\n> On Thu, Jul 09, 2026 at 03:19:52PM -0500, Justin Tobler wrote:\n> > > diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\n> > > index ea5eab4cf8..8ff92c5272 100644\n> > > --- a/builtin/pack-objects.c\n> > > +++ b/builtin/pack-objects.c\n> > > @@ -1909,7 +1909,7 @@ static int add_object_entry_from_bitmap(const struct object_id *oid,\n> > >  \t\treturn 0;\n> > >  \n> > >  \tcreate_object_entry(oid, type, name_hash, 0, 0, pack, offset);\n> > > -\treturn 1;\n> > > +\treturn 0;\n> > \n> > I wonder why this was even returning 1 to begin with? As you mentioned,\n> > the return value appears to be ignored anyways. I'm assuming it was\n> > signal that an object entry was created?\n> \n> The function is only called from a single location, and the return value\n> was completely ignored until this commit. It has always been this way\n> since the function was originally introduced in 6b8fda2db1\n> (pack-objects: use bitmaps when packing objects, 2013-12-21), so it\n> never seemed to have any purpose. The commit message doesn't mention\n> anything either.\n\nI think it was copying the semantics of its non-bitmap counterpart,\nadd_object_entry(). Of course nobody looks at that return value either!\n\nLong ago there were callers that cared about whether we actually created\nan entry, but I think the last one went away in 5379a5c5ee (Thin pack\ngeneration: optimization., 2006-04-05), which was quite some time ago.\n\nSo I think we could probably drop the return value from\nadd_object_entry() entirely (but of course we can't do the same for the\nbitmap variant, because of its use as a callback).\n\nI mention this mostly as answering Justin's \"I wonder why...\", but it\nmight be worth cleaning up add_object_entry() here, as its return value\nsemantics have diverged from add_object_entry_from_bitmap().\n\n-Peff\n"},{"id":"547814","messageId":"20260711075811.GC1457061@coredump.intra.peff.net","threadId":"65959","inReplyTo":"20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im","subject":"[PATCH v2 9/8?] pack-objects: drop unused return value from add_object_entry()","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-07-11T07:58:11Z","receivedAt":"2026-07-11T07:58:12Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 10:48:52AM +0200, Patrick Steinhardt wrote:\n\n> The series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n> 2026-07-06) with ps/odb-drop-whence at 8a7ad23e11 (odb: document object\n> info fields, 2026-07-02) merged into it.\n\nHere's a patch doing the cleanup I proposed upthread.\n\n-- >8 --\nSubject: pack-objects: drop unused return value from add_object_entry()\n\nThis function returns 0/1 to its caller to tell them whether we actually\nadded a new entry (or if we considered it redundant). But nobody has\nrelied on that behavior since 5379a5c5ee (Thin pack generation:\noptimization., 2006-04-05).\n\nThe extra return does not hurt much, but it recently became a bit more\nconfusing. We have a sister function, add_object_entry_from_bitmap(),\nwhich had the same return value semantics. That function recently\nchanged to always return 0 (not void, because it must conform to a\ncallback function interface). So now we have two related functions which\nboth return an \"int\" but with different semantics.\n\nLet's drop the unused \"int\" return from add_object_entry() entirely,\nwhich makes it more clear that the two functions have diverged.\n\nSigned-off-by: Jeff King <peff@peff.net>\n---\nI couldn't reference the commit by its id, since Junio has not yet\npicked up the v2 sent a few hours ago. ;)\n\n builtin/pack-objects.c | 9 ++++-----\n 1 file changed, 4 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 8ff92c5272..3673b14b89 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1867,16 +1867,16 @@ static const char no_closure_warning[] = N_(\n \"disabling bitmap writing, as some objects are not being packed\"\n );\n \n-static int add_object_entry(const struct object_id *oid, enum object_type type,\n-\t\t\t    const char *name, int exclude)\n+static void add_object_entry(const struct object_id *oid, enum object_type type,\n+\t\t\t     const char *name, int exclude)\n {\n \tstruct packed_git *found_pack = NULL;\n \toff_t found_offset = 0;\n \n \tdisplay_progress(progress_state, ++nr_seen);\n \n \tif (have_duplicate_entry(oid, exclude))\n-\t\treturn 0;\n+\t\treturn;\n \n \tif (!want_object_in_pack(oid, exclude, &found_pack, &found_offset)) {\n \t\t/* The pack is missing an object, so it will not have closure */\n@@ -1885,13 +1885,12 @@ static int add_object_entry(const struct object_id *oid, enum object_type type,\n \t\t\t\twarning(_(no_closure_warning));\n \t\t\twrite_bitmap_index = 0;\n \t\t}\n-\t\treturn 0;\n+\t\treturn;\n \t}\n \n \tcreate_object_entry(oid, type, pack_name_hash_fn(name),\n \t\t\t    exclude, name && no_try_delta(name),\n \t\t\t    found_pack, found_offset);\n-\treturn 1;\n }\n \n static int add_object_entry_from_bitmap(const struct object_id *oid,\n-- \n2.55.0.580.gbbcb530e9e\n\n"},{"id":"547815","messageId":"20260711080114.GA1470749@coredump.intra.peff.net","threadId":"65959","inReplyTo":"alFzja98avOoKjQE@com-79390","subject":"Re: [PATCH v2 3/8] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-07-11T08:01:14Z","receivedAt":"2026-07-11T08:01:15Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 03:34:53PM -0700, Taylor Blau wrote:\n\n> However, the remaining `show_objects_for_type()` callers from within\n> `traverse_bitmap_commit_list()` do *not* bother to inspect the return\n> value, despite taking in an arbitrary 'show_reachable_fn', which itself\n> may return a non-zero value.\n> \n> I guess this must be effectively OK in practice with respect to the\n> existing code for the same reason you indicate in the commit message\n> above, but we should change this function to *also* propagate non-zero\n> return values to eliminate the foot-gun completely.\n\nThe matching non-bitmap traverse_commit_list() does not allow aborting\nbased on callback returns, either. In fact, its callbacks return void!\n\nWhichever direction we go, those two should probably stay in sync (so\neither both should allow aborting early with a non-zero return, or both\nshould return void).\n\n-Peff\n"},{"id":"547845","messageId":"xmqq8q7hbhq7.fsf@gitster.g","threadId":"65959","inReplyTo":"20260711075811.GC1457061@coredump.intra.peff.net","subject":"Re: [PATCH v2 9/8?] pack-objects: drop unused return value from add_object_entry()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-11T16:42:40Z","receivedAt":"2026-07-11T16:42:43Z","isPatch":true,"body":"Jeff King <peff@peff.net> writes:\n\n> On Fri, Jul 10, 2026 at 10:48:52AM +0200, Patrick Steinhardt wrote:\n>\n>> The series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n>> 2026-07-06) with ps/odb-drop-whence at 8a7ad23e11 (odb: document object\n>> info fields, 2026-07-02) merged into it.\n>\n> Here's a patch doing the cleanup I proposed upthread.\n>\n> -- >8 --\n> Subject: pack-objects: drop unused return value from add_object_entry()\n>\n> This function returns 0/1 to its caller to tell them whether we actually\n> added a new entry (or if we considered it redundant). But nobody has\n> relied on that behavior since 5379a5c5ee (Thin pack generation:\n> optimization., 2006-04-05).\n>\n> The extra return does not hurt much, but it recently became a bit more\n> confusing. We have a sister function, add_object_entry_from_bitmap(),\n> which had the same return value semantics. That function recently\n> changed to always return 0 (not void, because it must conform to a\n> callback function interface). So now we have two related functions which\n> both return an \"int\" but with different semantics.\n>\n> Let's drop the unused \"int\" return from add_object_entry() entirely,\n> which makes it more clear that the two functions have diverged.\n>\n> Signed-off-by: Jeff King <peff@peff.net>\n> ---\n> I couldn't reference the commit by its id, since Junio has not yet\n> picked up the v2 sent a few hours ago. ;)\n\nHeh, if you do intend to make this a part of the series as 9/8, you\ncan just say \"earlier in the series\" with its title, and that should\nbe sufficient to identify which patch, as I never make a\nfast-forward merge when merging topics into integration branches\n(which means that Michael's \"git when-merged\" works well).\n\nAnd if we ever see v3 of this series, you and Patrick can work\ntogether to see if it makes sense to squash it in, or move it\nearlier in a series to as preliminary clean-up, etc.\n\nThanks.  I agree with the reasoning upthread that led to this\nchange.\n\n"},{"id":"547963","messageId":"alS1rus9thie7NiE@pks.im","threadId":"65959","inReplyTo":"20260711080114.GA1470749@coredump.intra.peff.net","subject":"Re: [PATCH v2 3/8] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T09:53:50Z","receivedAt":"2026-07-13T09:54:04Z","isPatch":true,"body":"On Sat, Jul 11, 2026 at 04:01:14AM -0400, Jeff King wrote:\n> On Fri, Jul 10, 2026 at 03:34:53PM -0700, Taylor Blau wrote:\n> \n> > However, the remaining `show_objects_for_type()` callers from within\n> > `traverse_bitmap_commit_list()` do *not* bother to inspect the return\n> > value, despite taking in an arbitrary 'show_reachable_fn', which itself\n> > may return a non-zero value.\n> > \n> > I guess this must be effectively OK in practice with respect to the\n> > existing code for the same reason you indicate in the commit message\n> > above, but we should change this function to *also* propagate non-zero\n> > return values to eliminate the foot-gun completely.\n> \n> The matching non-bitmap traverse_commit_list() does not allow aborting\n> based on callback returns, either. In fact, its callbacks return void!\n> \n> Whichever direction we go, those two should probably stay in sync (so\n> either both should allow aborting early with a non-zero return, or both\n> should return void).\n\nThat's fair. But adapting `traverse_commit_list()` requires tons of\nchanges all over the tree, so I'm inclined to rather leave both\n`traverse_bitmap_commit_list()` and `traverse_commit_list()` as-is.\nDoes that work for both of you?\n\nPatrick\n"},{"id":"547964","messageId":"alS1440iifvTvGKP@pks.im","threadId":"65959","inReplyTo":"alFxRvkfNgJRCQTB@com-79390","subject":"Re: [PATCH v2 1/8] odb/source-packed: improve lookup when enumerating objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T09:54:43Z","receivedAt":"2026-07-13T09:54:52Z","isPatch":true,"body":"On Fri, Jul 10, 2026 at 03:25:10PM -0700, Taylor Blau wrote:\n> On Fri, Jul 10, 2026 at 10:48:53AM +0200, Patrick Steinhardt wrote:\n> > Fix the issue by using `packed_object_info()` directly.\n> \n> What you wrote here makes sense to me insofar as I understand the\n> pluggable ODB code.\n> \n> However, I am confused by the way this function is written in general.\n> We use `bsearch_one_midx()` to locate the first possible MIDX position\n> in which an object matching the given prefix may exist, which is\n> sensible. However, we go from that position up to \"num\", where \"num\" is\n> the total number of objects in the MIDX!\n> \n> Functionally this is not incorrect as we will happily discard objects\n> that do not match the prefix. But it causes us to waste CPU cycles\n> repeatedly calling `match_hash()` (at least for the first byte of the\n> prefix) for objects that we know will match.\n\nThat's not quite true though, as we abort iteration as soon as\n`match_hash()` tells us that the prefix doesn't match anymore.\n\nOr do you mean that `num` should only be `m->num_objects` instead of\nalso iterating through `num_objects_in_base`? I have to admit that I'm\nalwas struggling with the chained MIDX. It's never quite clear to me\nwhether a given function cares about the complete chain or whether it\nreally only cares about a single MIDX.\n\nIn any case, this code ultimately derives from 3f5f1cff92 (midx:\nintroduce `bsearch_one_midx()`, 2024-08-06). If one squints a bit you\ncan see that it's still roughly in the same shape.\n\n> How often do we call this function with a prefix longer than a\n> single byte? I have no idea, but I would suspect that it makes up the\n> majority of calls. If we read the OID fanout chunk, we could narrow the\n> range that we enumerate through, and only compare the second byte\n> onwards of the given prefix, if one exists. In the single-byte prefix\n> case, this means that we shouldn't have to do any memory comparisons at\n> all.\n\nThe function is currently used to find unique prefixes and to\ndisambiguate object names. So whenever we either want to abbreviate a\nobject ID or in case we cant to figure out whether a given object ID\nprefix is unique we'll end up calling it.\n\nIf this logic is currently wrong (or at least wasteful) though I'd\npropose to fix this in a separate series, as it's been this way for\nquite a while.\n\nThanks!\n\nPatrick\n"},{"id":"548000","messageId":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH v3 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:24Z","receivedAt":"2026-07-13T14:41:34Z","isPatch":true,"body":"Hi,\n\nthis patch series introduces object filters to `odb_for_each_object()`.\nThe intent of this is to make `git cat-file --batch-all-objects` work\nwith pluggable object databases. Right now it doesn't because it reaches\ninto internals of the \"packed\" backend to efficiently handle bitmapped\nobjects.\n\nThe series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n2026-07-06) with ps/odb-drop-whence at 8a7ad23e11 (odb: document object\ninfo fields, 2026-07-02) merged into it.\n\nChanges in v3:\n  - Weave Peff's patch into the patch series.\n  - Link to v2: https://patch.msgid.link/20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im\n\nChanges in v2:\n  - Add another patch to drop the `_1()` prefixes that aren't required\n    anymore.\n  - Change the approach in `open_bitmap_for_source()` to also use a\n    `found` boolean instead of a confusing integer.\n  - Add some more explanations to commit messages.\n  - Link to v1: https://patch.msgid.link/20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im\n\nThanks!\n\nPatrick\n\n---\nJeff King (1):\n      pack-objects: drop unused return value from add_object_entry()\n\nPatrick Steinhardt (8):\n      odb/source-packed: improve lookup when enumerating objects\n      pack-bitmap: mark object filter as `const`\n      pack-bitmap: allow aborting iteration of bitmapped objects\n      pack-bitmap: iterate object sources when opening bitmaps\n      pack-bitmap: drop `_1` suffix from functions that open bitmaps\n      pack-bitmap: introduce function to open bitmap for a single source\n      odb: introduce object filters to `odb_for_each_object()`\n      builtin/cat-file: filter objects via object database\n\n builtin/cat-file.c     |  76 +++--------------------------\n builtin/pack-objects.c |  11 ++---\n builtin/rev-list.c     |   2 +-\n odb.h                  |  12 +++++\n odb/source-packed.c    |  77 ++++++++++++++++++++++++++---\n pack-bitmap.c          | 129 +++++++++++++++++++++++++++----------------------\n pack-bitmap.h          |  10 +++-\n 7 files changed, 175 insertions(+), 142 deletions(-)\n\nRange-diff versus v2:\n\n 1:  baf2adb012 =  1:  7c0dc1be0d odb/source-packed: improve lookup when enumerating objects\n 2:  57eecf3031 =  2:  2e5908c9c3 pack-bitmap: mark object filter as `const`\n -:  ---------- >  3:  f4d66ccfc6 pack-objects: drop unused return value from add_object_entry()\n 3:  92dd6a6f6e =  4:  af475654b8 pack-bitmap: allow aborting iteration of bitmapped objects\n 4:  92fe41577d =  5:  6ca42587c9 pack-bitmap: iterate object sources when opening bitmaps\n 5:  e5d59959e3 =  6:  f62c3bbc81 pack-bitmap: drop `_1` suffix from functions that open bitmaps\n 6:  ab3547ac2b =  7:  b2d25b6e9b pack-bitmap: introduce function to open bitmap for a single source\n 7:  026f21f522 =  8:  a5bf309bec odb: introduce object filters to `odb_for_each_object()`\n 8:  534b25c817 =  9:  600b15a907 builtin/cat-file: filter objects via object database\n\n---\nbase-commit: 3c8e2790f2ce15e8b5d4b4e6ced711b12649f32a\nchange-id: 20260708-pks-odb-for-each-object-filter-13286fa3523d\n\n"},{"id":"548001","messageId":"20260713-pks-odb-for-each-object-filter-v3-1-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 1/9] odb/source-packed: improve lookup when enumerating objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:25Z","receivedAt":"2026-07-13T14:41:36Z","isPatch":true,"body":"When iterating through objects of a packed source that have a specific\nprefix we do so via two different methods:\n\n  - When a multi-pack index is available we use that one to efficiently\n    loop through all objects.\n\n  - We then loop through all packfiles that aren't covered by a\n    multi-pack index.\n\nRegardless of which mechanism we use, we then iterate through all the\nobjects indexed by the respective data structure. Curiously though,\nwhile we use the indices for enumerating the objects, we completely\nignore it for the actual object lookup. Instead, we call into the\ngeneric `odb_source_read_object_info()` function, which will itself\nconsult the indices to figure out where the object in question even\nlives.\n\nThis has two consequences:\n\n  - It's inefficient, as we basically have to figure out the position of\n    the object a second time.\n\n  - It's subtly wrong, as it may now happen that a specific object will\n    be looked up via a different pack in case it exists multiple times.\n    This is unlikely to have any real-world consequences, but it's still\n    the wrong thing to do.\n\nFix the issue by using `packed_object_info()` directly. While at it,\nrename the `store` variable to `source`.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n odb/source-packed.c | 15 ++++++++-------\n 1 file changed, 8 insertions(+), 7 deletions(-)\n\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 0edea5356d..9cfa02b7a2 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -143,7 +143,7 @@ static bool should_exclude_pack(struct packed_git *p, enum odb_for_each_object_f\n }\n \n static int for_each_prefixed_object_in_midx(\n-\tstruct odb_source_packed *store,\n+\tstruct odb_source_packed *source,\n \tstruct multi_pack_index *m,\n \tconst struct odb_for_each_object_options *opts,\n \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n@@ -170,6 +170,7 @@ static int for_each_prefixed_object_in_midx(\n \t\t */\n \t\tfor (i = first; i < num; i++) {\n \t\t\tconst struct object_id *current = NULL;\n+\t\t\tstruct packed_git *pack;\n \t\t\tstruct object_id oid;\n \n \t\t\tcurrent = nth_midxed_object_oid(&oid, m, i);\n@@ -177,9 +178,8 @@ static int for_each_prefixed_object_in_midx(\n \t\t\tif (!match_hash(len, opts->prefix->hash, current->hash))\n \t\t\t\tbreak;\n \n-\t\t\tif (opts->flags) {\n+\t\t\tif (opts->flags || data->request) {\n \t\t\t\tuint32_t pack_id = nth_midxed_pack_int_id(m, i);\n-\t\t\t\tstruct packed_git *pack;\n \n \t\t\t\tif (prepare_midx_pack(m, pack_id)) {\n \t\t\t\t\tpack_errors = true;\n@@ -193,9 +193,9 @@ static int for_each_prefixed_object_in_midx(\n \n \t\t\tif (data->request) {\n \t\t\t\tstruct object_info oi = *data->request;\n+\t\t\t\toff_t offset = nth_midxed_offset(m, i);\n \n-\t\t\t\tret = odb_source_read_object_info(&store->base, current,\n-\t\t\t\t\t\t\t\t  &oi, 0);\n+\t\t\t\tret = packed_object_info(source, pack, offset, &oi);\n \t\t\t\tif (ret)\n \t\t\t\t\tgoto out;\n \n@@ -219,7 +219,7 @@ static int for_each_prefixed_object_in_midx(\n }\n \n static int for_each_prefixed_object_in_pack(\n-\tstruct odb_source_packed *store,\n+\tstruct odb_source_packed *source,\n \tstruct packed_git *p,\n \tconst struct odb_for_each_object_options *opts,\n \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n@@ -246,8 +246,9 @@ static int for_each_prefixed_object_in_pack(\n \n \t\tif (data->request) {\n \t\t\tstruct object_info oi = *data->request;\n+\t\t\toff_t offset = nth_packed_object_offset(p, i);\n \n-\t\t\tret = odb_source_read_object_info(&store->base, &oid, &oi, 0);\n+\t\t\tret = packed_object_info(source, p, offset, &oi);\n \t\t\tif (ret)\n \t\t\t\tgoto out;\n \n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548002","messageId":"20260713-pks-odb-for-each-object-filter-v3-2-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 2/9] pack-bitmap: mark object filter as `const`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:26Z","receivedAt":"2026-07-13T14:41:38Z","isPatch":true,"body":"The function `for_each_bitmapped_object()` accepts an optional object\nfilter. This filter is never modified by the function, but is not\ndeclared as `const`. Fix this.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 6 +++---\n pack-bitmap.h | 2 +-\n 2 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 35774b6f0c..a47c231632 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1976,7 +1976,7 @@ static void filter_bitmap_object_type(struct bitmap_index *bitmap_git,\n static int filter_bitmap(struct bitmap_index *bitmap_git,\n \t\t\t struct object_list *tip_objects,\n \t\t\t struct bitmap *to_filter,\n-\t\t\t struct list_objects_filter_options *filter)\n+\t\t\t const struct list_objects_filter_options *filter)\n {\n \tif (!filter || filter->choice == LOFC_DISABLED)\n \t\treturn 0;\n@@ -2027,7 +2027,7 @@ static int filter_bitmap(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int can_filter_bitmap(struct list_objects_filter_options *filter)\n+static int can_filter_bitmap(const struct list_objects_filter_options *filter)\n {\n \treturn !filter_bitmap(NULL, NULL, NULL, filter);\n }\n@@ -2058,7 +2058,7 @@ static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,\n }\n \n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n-\t\t\t      struct list_objects_filter_options *filter,\n+\t\t\t      const struct list_objects_filter_options *filter,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload)\n {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 19a8655457..47935eb24e 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -96,7 +96,7 @@ struct list_objects_filter_options;\n  * not supported, `0` otherwise.\n  */\n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n-\t\t\t      struct list_objects_filter_options *filter,\n+\t\t\t      const struct list_objects_filter_options *filter,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload);\n \n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548003","messageId":"20260713-pks-odb-for-each-object-filter-v3-3-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 3/9] pack-objects: drop unused return value from add_object_entry()","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:27Z","receivedAt":"2026-07-13T14:41:40Z","isPatch":true,"body":"From: Jeff King <peff@peff.net>\n\nThis function returns 0/1 to its caller to tell them whether we actually\nadded a new entry (or if we considered it redundant). But nobody has\nrelied on that behavior since 5379a5c5ee (Thin pack generation:\noptimization., 2006-04-05).\n\nThe extra return does not hurt much, but it is a bit confusing. We have\na sister function, add_object_entry_from_bitmap(), which has the same\nreturn value semantics. That function is about to change to always return\n0 (not void, because it must conform to a callback function interface).\nSo with that change, we'd have two related functions which both return\nan \"int\" but with different semantics.\n\nLet's drop the unused \"int\" return from add_object_entry() entirely,\nwhich makes it more clear that the two functions have diverged.\n\nSigned-off-by: Jeff King <peff@peff.net>\n[ps: slightly massaged the commit message]\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/pack-objects.c | 9 ++++-----\n 1 file changed, 4 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex ea5eab4cf8..188c4f6d4b 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1867,8 +1867,8 @@ static const char no_closure_warning[] = N_(\n \"disabling bitmap writing, as some objects are not being packed\"\n );\n \n-static int add_object_entry(const struct object_id *oid, enum object_type type,\n-\t\t\t    const char *name, int exclude)\n+static void add_object_entry(const struct object_id *oid, enum object_type type,\n+\t\t\t     const char *name, int exclude)\n {\n \tstruct packed_git *found_pack = NULL;\n \toff_t found_offset = 0;\n@@ -1876,7 +1876,7 @@ static int add_object_entry(const struct object_id *oid, enum object_type type,\n \tdisplay_progress(progress_state, ++nr_seen);\n \n \tif (have_duplicate_entry(oid, exclude))\n-\t\treturn 0;\n+\t\treturn;\n \n \tif (!want_object_in_pack(oid, exclude, &found_pack, &found_offset)) {\n \t\t/* The pack is missing an object, so it will not have closure */\n@@ -1885,13 +1885,12 @@ static int add_object_entry(const struct object_id *oid, enum object_type type,\n \t\t\t\twarning(_(no_closure_warning));\n \t\t\twrite_bitmap_index = 0;\n \t\t}\n-\t\treturn 0;\n+\t\treturn;\n \t}\n \n \tcreate_object_entry(oid, type, pack_name_hash_fn(name),\n \t\t\t    exclude, name && no_try_delta(name),\n \t\t\t    found_pack, found_offset);\n-\treturn 1;\n }\n \n static int add_object_entry_from_bitmap(const struct object_id *oid,\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548004","messageId":"20260713-pks-odb-for-each-object-filter-v3-4-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 4/9] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:28Z","receivedAt":"2026-07-13T14:41:42Z","isPatch":true,"body":"In a subsequent commit we'll lift iteration of bitmapped objects into\nthe \"packed\" backend and make it accessible via `odb_for_each_object()`.\nThe calling convention for that function is that the callback may return\na non-zero exit code, and if so we'll abort iteration. This is currently\nimpossible to realize though, as `for_each_bitmapped_object()` will\nignore any return value and just churn through all objects completely.\n\nThis doesn't matter to the callers of `for_each_bitmapped_object()`, as\nthere's only one of them in git-cat-file(1), and the callbacks we pass\nalways return zero. But once we move the logic into the generic\ninfrastructure it becomes a latent bug waiting to happen.\n\nRefactor the code so that the return value of the `show_reach` callback\nis not ignored anymore. Instead, returning a non-zero value will cause\nus to abort iteration in both `show_objects_for_type()` and in\n`for_each_bitmapped_object()`.\n\nNote though that there's a second user of `show_objects_for_type()` with\n`traverse_bitmap_commit_list()`, and that function does indeed invoke\ncallbacks that may return non-zero. This non-zero return value never had\nany effect at all though, and the callbacks that return non-zero values\nare only ever invoked via `traverse_bitmap_commit_list()`. Consequently,\nwe adapt them to always return 0.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/pack-objects.c |  2 +-\n builtin/rev-list.c     |  2 +-\n pack-bitmap.c          | 31 +++++++++++++++++++++----------\n pack-bitmap.h          |  3 ++-\n 4 files changed, 25 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 188c4f6d4b..3673b14b89 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1908,7 +1908,7 @@ static int add_object_entry_from_bitmap(const struct object_id *oid,\n \t\treturn 0;\n \n \tcreate_object_entry(oid, type, name_hash, 0, 0, pack, offset);\n-\treturn 1;\n+\treturn 0;\n }\n \n struct pbase_tree_cache {\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 8f63003709..02818b81c6 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -486,7 +486,7 @@ static int show_object_fast(\n \tvoid *payload UNUSED)\n {\n \tfprintf(stdout, \"%s\\n\", oid_to_hex(oid));\n-\treturn 1;\n+\treturn 0;\n }\n \n static void print_disk_usage(off_t size)\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex a47c231632..eda38a5433 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1695,7 +1695,7 @@ static void init_type_iterator(struct ewah_or_iterator *it,\n \t}\n }\n \n-static void show_objects_for_type(\n+static int show_objects_for_type(\n \tstruct bitmap_index *bitmap_git,\n \tstruct bitmap *objects,\n \tenum object_type object_type,\n@@ -1704,6 +1704,7 @@ static void show_objects_for_type(\n {\n \tsize_t i = 0;\n \tuint32_t offset;\n+\tint ret;\n \n \tstruct ewah_or_iterator it;\n \teword_t filter;\n@@ -1749,11 +1750,17 @@ static void show_objects_for_type(\n \n \t\t\thash = bitmap_name_hash(bitmap_git, index_pos);\n \n-\t\t\tshow_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n+\t\t\tret = show_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n+\t\t\tif (ret)\n+\t\t\t\tgoto out;\n \t\t}\n \t}\n \n+\tret = 0;\n+\n+out:\n \tewah_or_iterator_release(&it);\n+\treturn ret;\n }\n \n static int in_bitmapped_pack(struct bitmap_index *bitmap_git,\n@@ -2062,6 +2069,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload)\n {\n+\tconst enum object_type types[] = {\n+\t\tOBJ_COMMIT,\n+\t\tOBJ_TREE,\n+\t\tOBJ_BLOB,\n+\t\tOBJ_TAG,\n+\t};\n \tstruct bitmap *filtered_bitmap = NULL;\n \tuint32_t objects_nr;\n \tsize_t full_word_count;\n@@ -2086,14 +2099,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\tgoto out;\n \t}\n \n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_COMMIT, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_TREE, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_BLOB, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_TAG, show_reach, payload);\n+\tfor (size_t i = 0; i < ARRAY_SIZE(types); i++) {\n+\t\tret = show_objects_for_type(bitmap_git, filtered_bitmap,\n+\t\t\t\t\t    types[i], show_reach, payload);\n+\t\tif (ret)\n+\t\t\tgoto out;\n+\t}\n \n \tret = 0;\n out:\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 47935eb24e..ae8dc491ac 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -93,7 +93,8 @@ struct list_objects_filter_options;\n /*\n  * Filter bitmapped objects and iterate through all resulting objects,\n  * executing `show_reach` for each of them. Returns `-1` in case the filter is\n- * not supported, `0` otherwise.\n+ * not supported, `0` otherwise. Aborts iteration and bubbles up the return\n+ * value in case `show_reach()` returns non-zero.\n  */\n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\t\t      const struct list_objects_filter_options *filter,\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548005","messageId":"20260713-pks-odb-for-each-object-filter-v3-5-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 5/9] pack-bitmap: iterate object sources when opening bitmaps","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:29Z","receivedAt":"2026-07-13T14:41:45Z","isPatch":true,"body":"When opening a bitmap for a repository we perform two steps:\n\n  - We first look for a multi-pack index bitmap in any of the object\n    sources connected to the repository.\n\n  - We then look for a packfile bitmap in any of the packfiles of any of\n    the object sources.\n\nBoth of these steps thus iterate through object sources themselves, one\nvia `odb_prepare_alternates()` and one via `repo_for_each_pack()`. This\nlayout makes it hard to introduce a way to open the bitmap of one\nspecific object source, which is functionality that we'll require in a\nsubsequent commit.\n\nReverse the loop so that we instead loop through all sources in the\nouter loop, and then for each source we try to load its bitmap via\neither the multi-pack index or via a packfile.\n\nNote that this changes the precedence of bitmaps in one specific edge\ncase: when an earlier object source only has a packfile bitmap, but a\nlater source has a multi-pack index bitmap, we now pick the packfile\nbitmap of the earlier source. Previously, a multi-pack index bitmap from\nany source would have taken precedence over all packfile bitmaps. Given\nthat object sources are ordered such that the local source comes first,\nthis arguably is an improvement, as we now prefer local bitmaps over\nbitmaps in alternates. Furthermore, we already warn about repositories\nthat have multiple bitmaps, so this setup is broken and thus arguably\nnot worth worrying about too much.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 69 +++++++++++++++++++++++++++--------------------------------\n 1 file changed, 31 insertions(+), 38 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex eda38a5433..e32795a595 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -680,60 +680,53 @@ static int load_bitmap(struct repository *r, struct bitmap_index *bitmap_git,\n \treturn 0;\n }\n \n-static int open_pack_bitmap(struct repository *r,\n-\t\t\t    struct bitmap_index *bitmap_git)\n+static int open_bitmap_for_source(struct odb_source_packed *source,\n+\t\t\t\t  struct bitmap_index *bitmap_git)\n {\n-\tstruct packed_git *p;\n-\tint ret = -1;\n+\tstruct multi_pack_index *midx = get_multi_pack_index(source);\n+\tstruct packfile_list_entry *e;\n+\tbool found = false;\n \n-\trepo_for_each_pack(r, p) {\n-\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n-\t\t\tret = 0;\n-\t\t\t/*\n-\t\t\t * The only reason to keep looking is to report\n-\t\t\t * duplicates.\n-\t\t\t */\n-\t\t\tif (!trace2_is_enabled())\n-\t\t\t\tbreak;\n-\t\t}\n+\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n+\t\tfound = true;\n+\n+\tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n+\t\t/*\n+\t\t * When tracing is enabled we want to keep looking to report\n+\t\t * duplicates even if we have already found a bitmap.\n+\t\t */\n+\t\tif (found && !trace2_is_enabled())\n+\t\t\tbreak;\n+\n+\t\tif (!open_pack_bitmap_1(bitmap_git, e->pack))\n+\t\t\tfound = true;\n \t}\n \n-\treturn ret;\n+\treturn found ? 0 : -1;\n }\n \n-static int open_midx_bitmap(struct repository *r,\n-\t\t\t    struct bitmap_index *bitmap_git)\n+static int open_bitmap(struct repository *r,\n+\t\t       struct bitmap_index *bitmap_git)\n {\n \tstruct odb_source *source;\n-\tint ret = -1;\n+\tbool found = false;\n \n \tassert(!bitmap_git->map);\n \n \todb_prepare_alternates(r->objects);\n \tfor (source = r->objects->sources; source; source = source->next) {\n \t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\tstruct multi_pack_index *midx = get_multi_pack_index(files->packed);\n-\t\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\tret = 0;\n-\t}\n-\treturn ret;\n-}\n-\n-static int open_bitmap(struct repository *r,\n-\t\t       struct bitmap_index *bitmap_git)\n-{\n-\tint found;\n \n-\tassert(!bitmap_git->map);\n+\t\tif (!open_bitmap_for_source(files->packed, bitmap_git))\n+\t\t\tfound = true;\n \n-\tfound = !open_midx_bitmap(r, bitmap_git);\n-\n-\t/*\n-\t * these will all be skipped if we opened a midx bitmap; but run it\n-\t * anyway if tracing is enabled to report the duplicates\n-\t */\n-\tif (!found || trace2_is_enabled())\n-\t\tfound |= !open_pack_bitmap(r, bitmap_git);\n+\t\t/*\n+\t\t * The only reason to keep looking after having found a bitmap\n+\t\t * is to report duplicates.\n+\t\t */\n+\t\tif (found && !trace2_is_enabled())\n+\t\t\tbreak;\n+\t}\n \n \treturn found ? 0 : -1;\n }\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548006","messageId":"20260713-pks-odb-for-each-object-filter-v3-6-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 6/9] pack-bitmap: drop `_1` suffix from functions that open bitmaps","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:30Z","receivedAt":"2026-07-13T14:41:47Z","isPatch":true,"body":"In the preceding commit we've refactored how we open bitmaps. As part of\nthe refactoring we have consolidated `open_pack_bitmap()` as well as\n`open_midx_bitmap()` into `open_bitmap_for_source()`. Consequently, we\nonly have their `open_pack_bitmap_1()` and `open_midx_bitmap_1()`\nvariants left over, where the `_1` suffix doesn't really make much sense\nanymore.\n\nDrop the suffix.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex e32795a595..72c8ae3228 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -460,8 +460,8 @@ char *pack_bitmap_filename(struct packed_git *p)\n \treturn xstrfmt(\"%.*s.bitmap\", (int)len, p->pack_name);\n }\n \n-static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n-\t\t\t      struct multi_pack_index *midx)\n+static int open_midx_bitmap(struct bitmap_index *bitmap_git,\n+\t\t\t    struct multi_pack_index *midx)\n {\n \tstruct stat st;\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n@@ -539,7 +539,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n+static int open_pack_bitmap(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n {\n \tint fd;\n \tstruct stat st;\n@@ -603,7 +603,7 @@ static int load_reverse_index(struct repository *r, struct bitmap_index *bitmap_\n \n \t\t/*\n \t\t * The multi-pack-index's .rev file is already loaded via\n-\t\t * open_pack_bitmap_1().\n+\t\t * open_pack_bitmap().\n \t\t *\n \t\t * But we still need to open the individual pack .rev files,\n \t\t * since we will need to make use of them in pack-objects.\n@@ -687,7 +687,7 @@ static int open_bitmap_for_source(struct odb_source_packed *source,\n \tstruct packfile_list_entry *e;\n \tbool found = false;\n \n-\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n+\tif (midx && !open_midx_bitmap(bitmap_git, midx))\n \t\tfound = true;\n \n \tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n@@ -698,7 +698,7 @@ static int open_bitmap_for_source(struct odb_source_packed *source,\n \t\tif (found && !trace2_is_enabled())\n \t\t\tbreak;\n \n-\t\tif (!open_pack_bitmap_1(bitmap_git, e->pack))\n+\t\tif (!open_pack_bitmap(bitmap_git, e->pack))\n \t\t\tfound = true;\n \t}\n \n@@ -746,7 +746,7 @@ struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx)\n {\n \tstruct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git));\n \n-\tif (!open_midx_bitmap_1(bitmap_git, midx))\n+\tif (!open_midx_bitmap(bitmap_git, midx))\n \t\treturn bitmap_git;\n \n \tfree_bitmap_index(bitmap_git);\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548007","messageId":"20260713-pks-odb-for-each-object-filter-v3-7-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 7/9] pack-bitmap: introduce function to open bitmap for a single source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:31Z","receivedAt":"2026-07-13T14:41:50Z","isPatch":true,"body":"The function `prepare_bitmap_git()` opens the first bitmap it can find\nin any of the object sources connected to the repository. In a\nsubsequent commit, the \"packed\" object database backend will learn to\nuse bitmaps to answer object filters when enumerating objects. That\nbackend operates on a single object source though, so using a bitmap\nthat potentially belongs to a different source would be wrong:\n\n  - The source would yield objects that are not part of the source\n    itself.\n\n  - The object source info would be attributed to the wrong source.\n\n  - With multiple sources, each source would enumerate the same bitmap\n    another time.\n\nIntroduce a new function `prepare_source_bitmap_git()` that only opens\nbitmaps belonging to the given object source.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 12 ++++++++++++\n pack-bitmap.h |  2 ++\n 2 files changed, 14 insertions(+)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 72c8ae3228..09ba15d26b 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -753,6 +753,18 @@ struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx)\n \treturn NULL;\n }\n \n+struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source)\n+{\n+\tstruct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git));\n+\n+\tif (!open_bitmap_for_source(source, bitmap_git) &&\n+\t    !load_bitmap(source->base.odb->repo, bitmap_git, 0))\n+\t\treturn bitmap_git;\n+\n+\tfree_bitmap_index(bitmap_git);\n+\treturn NULL;\n+}\n+\n int bitmap_index_contains_pack(struct bitmap_index *bitmap, struct packed_git *pack)\n {\n \tfor (; bitmap; bitmap = bitmap->base) {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex ae8dc491ac..9f20fb6e56 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -9,6 +9,7 @@\n #include \"string-list.h\"\n \n struct commit;\n+struct odb_source_packed;\n struct repository;\n struct rev_info;\n \n@@ -68,6 +69,7 @@ struct bitmapped_pack {\n \n struct bitmap_index *prepare_bitmap_git(struct repository *r);\n struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx);\n+struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source);\n \n /*\n  * Given a bitmap index, determine whether it contains the pack either directly\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548008","messageId":"20260713-pks-odb-for-each-object-filter-v3-8-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 8/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:32Z","receivedAt":"2026-07-13T14:41:53Z","isPatch":true,"body":"The function `for_each_bitmapped_object()` can be used to iterate\nthrough all objects covered by a bitmap. The benefit of this function is\nthat it allows the caller to efficiently handle some object filters. For\nexample, this can be used to filter out objects of a specific type with\nsome simple bitmap operations. But callers are currently required to\nmanually wire up the use of bitmaps though, and to do so they have to\nreach into internals of a given object database source.\n\nIntroduce a new `struct odb_for_each_object_options::filter` field so\nthat the interface becomes generic. When set, then a backend may\noptionally use the filter to skip some objects that it would have\notherwise yielded.\n\nNote that the respective backends are free to ignore this field if they\ncannot meaningfully optimize for a given filter, and consequently\ncallers need to verify whether they actually want the returned objects.\nWhile annoying, we cannot easily lift this restriction anyway as the\nobject filter infrastructure supports some filters that cannot be\nanswered by the object database alone.\n\nAn alternative might be to limit the filters to only those that _can_ be\nanswered by backends. But ultimately, the filters that can be answered\nefficiently by the \"packed\" backend are completely disjunct from those\nthat can be answered by the \"loose\" backend, and consequently the set of\nfilters supported by all backends would be empty. Furthermore, it would\nrequire us to make assumptions about capabilities of future backends,\nwhich may be able to efficiently handle more filters than current ones.\nSo in the end, this alternative would only limit us artificially.\n\nImplement the logic for the \"packed\" source. Note that we use the new\nfunction `prepare_source_bitmap_git()` to open the bitmap: as the\nbackend operates on a single object source, we must only use bitmaps\nthat belong to that specific source. Otherwise we might yield objects\nthat are not part of the source at all, and with multiple sources we\nwould enumerate the same bitmap once per source.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n odb.h               | 12 +++++++++++\n odb/source-packed.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n pack-bitmap.c       |  3 +--\n pack-bitmap.h       |  3 +++\n 4 files changed, 78 insertions(+), 2 deletions(-)\n\ndiff --git a/odb.h b/odb.h\nindex a1e222f605..67d0b34942 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -8,6 +8,7 @@\n #include \"thread-utils.h\"\n \n struct cached_object_entry;\n+struct list_objects_filter_options;\n struct odb_source_inmemory;\n struct packed_git;\n struct repository;\n@@ -490,6 +491,17 @@ struct odb_for_each_object_options {\n \t */\n \tconst struct object_id *prefix;\n \tsize_t prefix_hex_len;\n+\n+\t/*\n+\t * Optional object filter that allows backends to skip yielding\n+\t * objects that are excluded by the filter as an optimization. The\n+\t * filter is a best-effort hint: backends may use it to skip\n+\t * excluded objects (e.g. by consulting a reachability bitmap), but\n+\t * are also free to ignore it entirely and yield every object. As a\n+\t * consequence, callers must re-apply the filter on yielded objects\n+\t * if they require strict filtering semantics.\n+\t */\n+\tconst struct list_objects_filter_options *filter;\n };\n \n /*\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 9cfa02b7a2..4777395053 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -3,11 +3,13 @@\n #include \"chdir-notify.h\"\n #include \"dir.h\"\n #include \"git-zlib.h\"\n+#include \"list-objects-filter-options.h\"\n #include \"mergesort.h\"\n #include \"midx.h\"\n #include \"odb/source-packed.h\"\n #include \"odb/streaming.h\"\n #include \"packfile.h\"\n+#include \"pack-bitmap.h\"\n \n static int find_pack_entry(struct odb_source_packed *store,\n \t\t\t   const struct object_id *oid,\n@@ -315,6 +317,37 @@ static int odb_source_packed_for_each_prefixed_object(\n \treturn ret;\n }\n \n+struct bitmapped_for_each_object_data {\n+\tstruct odb_source_packed *packed;\n+\tconst struct object_info *request;\n+\tconst struct odb_for_each_object_options *opts;\n+\todb_for_each_object_cb cb;\n+\tvoid *cb_data;\n+};\n+\n+static int bitmapped_for_each_object(const struct object_id *oid,\n+\t\t\t\t     enum object_type type UNUSED,\n+\t\t\t\t     int flags UNUSED,\n+\t\t\t\t     uint32_t hash UNUSED,\n+\t\t\t\t     struct packed_git *pack,\n+\t\t\t\t     off_t offset,\n+\t\t\t\t     void *cb_data)\n+{\n+\tstruct bitmapped_for_each_object_data *data = cb_data;\n+\n+\tif (should_exclude_pack(pack, data->opts->flags))\n+\t\treturn 0;\n+\n+\tif (data->request) {\n+\t\tstruct object_info oi = *data->request;\n+\t\tif (packed_object_info(data->packed, pack, offset, &oi) < 0)\n+\t\t\treturn -1;\n+\t\treturn data->cb(oid, &oi, data->cb_data);\n+\t}\n+\n+\treturn data->cb(oid, NULL, data->cb_data);\n+}\n+\n static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\t\t\t\t     const struct object_info *request,\n \t\t\t\t\t     odb_for_each_object_cb cb,\n@@ -328,12 +361,33 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\t.cb = cb,\n \t\t.cb_data = cb_data,\n \t};\n+\tstruct bitmap_index *bitmap = NULL;\n \tstruct packfile_list_entry *e;\n \tint pack_errors = 0, ret;\n \n \tif (opts->prefix)\n \t\treturn odb_source_packed_for_each_prefixed_object(packed, opts, &data);\n \n+\tif (opts->filter &&\n+\t    opts->filter->choice != LOFC_DISABLED &&\n+\t    can_filter_bitmap(opts->filter))\n+\t\tbitmap = prepare_bitmap_git_for_source(packed);\n+\tif (bitmap) {\n+\t\tstruct bitmapped_for_each_object_data bitmap_data = {\n+\t\t\t.packed = packed,\n+\t\t\t.request = request,\n+\t\t\t.opts = opts,\n+\t\t\t.cb = cb,\n+\t\t\t.cb_data = cb_data,\n+\t\t};\n+\n+\t\tret = for_each_bitmapped_object(bitmap, opts->filter,\n+\t\t\t\t\t\tbitmapped_for_each_object,\n+\t\t\t\t\t\t&bitmap_data);\n+\t\tif (ret)\n+\t\t\tgoto out;\n+\t}\n+\n \tpacked->skip_mru_updates = true;\n \n \tfor (e = packfile_store_get_packs(packed); e; e = e->next) {\n@@ -342,6 +396,13 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\tif (should_exclude_pack(p, opts->flags))\n \t\t\tcontinue;\n \n+\t\t/*\n+\t\t * Objects covered by the bitmap have already been yielded\n+\t\t * above; skip them here to avoid duplicates.\n+\t\t */\n+\t\tif (bitmap && bitmap_index_contains_pack(bitmap, p))\n+\t\t\tcontinue;\n+\n \t\tif (open_pack_index(p)) {\n \t\t\tpack_errors = 1;\n \t\t\tcontinue;\n@@ -357,6 +418,7 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \n out:\n \tpacked->skip_mru_updates = false;\n+\tfree_bitmap_index(bitmap);\n \n \tif (!ret && pack_errors)\n \t\tret = -1;\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 09ba15d26b..f55a0859ea 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -2039,12 +2039,11 @@ static int filter_bitmap(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int can_filter_bitmap(const struct list_objects_filter_options *filter)\n+bool can_filter_bitmap(const struct list_objects_filter_options *filter)\n {\n \treturn !filter_bitmap(NULL, NULL, NULL, filter);\n }\n \n-\n static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,\n \t\t\t\t\t      struct bitmap *result)\n {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 9f20fb6e56..1385027c1f 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -92,6 +92,9 @@ int test_bitmap_pseudo_merge_objects(struct repository *r, uint32_t n);\n \n struct list_objects_filter_options;\n \n+/* Check whether the filter can be computed via the bitmap. */\n+bool can_filter_bitmap(const struct list_objects_filter_options *filter);\n+\n /*\n  * Filter bitmapped objects and iterate through all resulting objects,\n  * executing `show_reach` for each of them. Returns `-1` in case the filter is\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548009","messageId":"20260713-pks-odb-for-each-object-filter-v3-9-b3c65c641073@pks.im","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"[PATCH v3 9/9] builtin/cat-file: filter objects via object database","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-13T14:41:33Z","receivedAt":"2026-07-13T14:41:55Z","isPatch":true,"body":"When batching all objects, git-cat-file(1) reaches into the internals of\nthe object database and manually manages bitmaps to apply object\nfilters. This creates coupling between the command and the internals of\nthe respective backend.\n\nRefactor git-cat-file(1) to use the new object filter option when\nbatching all objects. This significantly simplifies the logic and\nensures that we don't have to reach into internals of the \"files\" source\nanymore.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/cat-file.c | 76 +++++-------------------------------------------------\n 1 file changed, 7 insertions(+), 69 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex b4b99a73da..1458dd76d6 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -20,7 +20,6 @@\n #include \"userdiff.h\"\n #include \"oid-array.h\"\n #include \"packfile.h\"\n-#include \"pack-bitmap.h\"\n #include \"object-file.h\"\n #include \"object-name.h\"\n #include \"odb.h\"\n@@ -844,28 +843,6 @@ static int batch_one_object_oi(const struct object_id *oid,\n \treturn payload->callback(oid, NULL, 0, payload->payload);\n }\n \n-static int batch_one_object_packed(const struct object_id *oid,\n-\t\t\t\t   struct packed_git *pack,\n-\t\t\t\t   uint32_t pos,\n-\t\t\t\t   void *_payload)\n-{\n-\tstruct for_each_object_payload *payload = _payload;\n-\treturn payload->callback(oid, pack, nth_packed_object_offset(pack, pos),\n-\t\t\t\t payload->payload);\n-}\n-\n-static int batch_one_object_bitmapped(const struct object_id *oid,\n-\t\t\t\t      enum object_type type UNUSED,\n-\t\t\t\t      int flags UNUSED,\n-\t\t\t\t      uint32_t hash UNUSED,\n-\t\t\t\t      struct packed_git *pack,\n-\t\t\t\t      off_t offset,\n-\t\t\t\t      void *_payload)\n-{\n-\tstruct for_each_object_payload *payload = _payload;\n-\treturn payload->callback(oid, pack, offset, payload->payload);\n-}\n-\n static void batch_each_object(struct batch_options *opt,\n \t\t\t      for_each_object_fn callback,\n \t\t\t      unsigned flags,\n@@ -875,56 +852,17 @@ static void batch_each_object(struct batch_options *opt,\n \t\t.callback = callback,\n \t\t.payload = _payload,\n \t};\n+\tstruct odb_source_info source_info;\n+\tstruct object_info oi = {\n+\t\t.source_infop = &source_info,\n+\t};\n \tstruct odb_for_each_object_options opts = {\n \t\t.flags = flags,\n+\t\t.filter = &opt->objects_filter,\n \t};\n-\tstruct bitmap_index *bitmap = NULL;\n-\tstruct odb_source *source;\n-\n-\t/*\n-\t * TODO: we still need to tap into implementation details of the object\n-\t * database sources. Ideally, we should extend `odb_for_each_object()`\n-\t * to handle object filters itself so that we can move the filtering\n-\t * logic into the individual sources.\n-\t */\n-\todb_prepare_alternates(the_repository->objects);\n-\tfor (source = the_repository->objects->sources; source; source = source->next) {\n-\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\tint ret = odb_source_for_each_object(&files->loose->base, NULL, batch_one_object_oi,\n-\t\t\t\t\t\t     &payload, &opts);\n-\t\tif (ret)\n-\t\t\tbreak;\n-\t}\n-\n-\tif (opt->objects_filter.choice != LOFC_DISABLED &&\n-\t    (bitmap = prepare_bitmap_git(the_repository)) &&\n-\t    !for_each_bitmapped_object(bitmap, &opt->objects_filter,\n-\t\t\t\t       batch_one_object_bitmapped, &payload)) {\n-\t\tstruct packed_git *pack;\n-\n-\t\trepo_for_each_pack(the_repository, pack) {\n-\t\t\tif (bitmap_index_contains_pack(bitmap, pack) ||\n-\t\t\t    open_pack_index(pack))\n-\t\t\t\tcontinue;\n-\t\t\tfor_each_object_in_pack(pack, batch_one_object_packed,\n-\t\t\t\t\t\t&payload, flags);\n-\t\t}\n-\t} else {\n-\t\tstruct odb_source_info source_info;\n-\t\tstruct object_info oi = {\n-\t\t\t.source_infop = &source_info,\n-\t\t};\n-\n-\t\tfor (source = the_repository->objects->sources; source; source = source->next) {\n-\t\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\t\tint ret = odb_source_for_each_object(&files->packed->base, &oi,\n-\t\t\t\t\t\t\t     batch_one_object_oi, &payload, &opts);\n-\t\t\tif (ret)\n-\t\t\t\tbreak;\n-\t\t}\n-\t}\n \n-\tfree_bitmap_index(bitmap);\n+\todb_for_each_object_ext(the_repository->objects, &oi,\n+\t\t\t\tbatch_one_object_oi, &payload, &opts);\n }\n \n static int batch_objects(struct batch_options *opt)\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548068","messageId":"alWx1wj1bc48g11X@com-79390","threadId":"65959","inReplyTo":"alS1440iifvTvGKP@pks.im","subject":"Re: [PATCH v2 1/8] odb/source-packed: improve lookup when enumerating objects","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-14T03:49:43Z","receivedAt":"2026-07-14T03:49:47Z","isPatch":true,"body":"On Mon, Jul 13, 2026 at 11:54:43AM +0200, Patrick Steinhardt wrote:\n> On Fri, Jul 10, 2026 at 03:25:10PM -0700, Taylor Blau wrote:\n> > On Fri, Jul 10, 2026 at 10:48:53AM +0200, Patrick Steinhardt wrote:\n> > > Fix the issue by using `packed_object_info()` directly.\n> >\n> > What you wrote here makes sense to me insofar as I understand the\n> > pluggable ODB code.\n> >\n> > However, I am confused by the way this function is written in general.\n> > We use `bsearch_one_midx()` to locate the first possible MIDX position\n> > in which an object matching the given prefix may exist, which is\n> > sensible. However, we go from that position up to \"num\", where \"num\" is\n> > the total number of objects in the MIDX!\n> >\n> > Functionally this is not incorrect as we will happily discard objects\n> > that do not match the prefix. But it causes us to waste CPU cycles\n> > repeatedly calling `match_hash()` (at least for the first byte of the\n> > prefix) for objects that we know will match.\n>\n> That's not quite true though, as we abort iteration as soon as\n> `match_hash()` tells us that the prefix doesn't match anymore.\n\nRight, we neither iterate through more objects than necessary once we\nknow that `match_hash()` will stop returning true, nor do we emit\nobjects that don't actually match the prefix.\n\nWhat I was trying to say above is that in the special case where our\nprefix is a single byte long, we don't have to call `match_hash()` at\n*all*, since we can enumerate just the portion of the fanout for that\nspecific byte, and we know that all such entries will match.\n\n> Or do you mean that `num` should only be `m->num_objects` instead of\n> also iterating through `num_objects_in_base`? I have to admit that I'm\n> alwas struggling with the chained MIDX. It's never quite clear to me\n> whether a given function cares about the complete chain or whether it\n> really only cares about a single MIDX.\n\nIf the goal is to yield all such objects that match the prefix, then we\nneed to enumerate each layer. The analogy that I have had in my head\nwhile working on these is that they are the same conceptually as the\nincremental commit-graph format.\n\nThanks,\nTaylor\n"},{"id":"548069","messageId":"alWz_gRs_D0Y0aOy@com-79390","threadId":"65959","inReplyTo":"alS1rus9thie7NiE@pks.im","subject":"Re: [PATCH v2 3/8] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-14T03:58:54Z","receivedAt":"2026-07-14T03:58:59Z","isPatch":true,"body":"On Mon, Jul 13, 2026 at 11:53:50AM +0200, Patrick Steinhardt wrote:\n> On Sat, Jul 11, 2026 at 04:01:14AM -0400, Jeff King wrote:\n> > On Fri, Jul 10, 2026 at 03:34:53PM -0700, Taylor Blau wrote:\n> >\n> > > However, the remaining `show_objects_for_type()` callers from within\n> > > `traverse_bitmap_commit_list()` do *not* bother to inspect the return\n> > > value, despite taking in an arbitrary 'show_reachable_fn', which itself\n> > > may return a non-zero value.\n> > >\n> > > I guess this must be effectively OK in practice with respect to the\n> > > existing code for the same reason you indicate in the commit message\n> > > above, but we should change this function to *also* propagate non-zero\n> > > return values to eliminate the foot-gun completely.\n> >\n> > The matching non-bitmap traverse_commit_list() does not allow aborting\n> > based on callback returns, either. In fact, its callbacks return void!\n> >\n> > Whichever direction we go, those two should probably stay in sync (so\n> > either both should allow aborting early with a non-zero return, or both\n> > should return void).\n>\n> That's fair. But adapting `traverse_commit_list()` requires tons of\n> changes all over the tree, so I'm inclined to rather leave both\n> `traverse_bitmap_commit_list()` and `traverse_commit_list()` as-is.\n> Does that work for both of you?\n\nI think that it's fine to leave it as-is for the purpose of this series,\nthough I would like to address it.\n\nI don't think we need to adapt `traverse_commit_list()`, though. We can\ngo in the other direction Peff suggested, which would be to split the\ncallback type used by `for_each_bitmapped_object()` from\n`show_reachable_fn`, keep the former abortable, and make the latter\nreturn void.\n\nThat keeps `traverse_bitmap_commit_list()` in sync with\n`traverse_commit_list()` without changing the non-bitmap traversal\nmachinery. I have a small two-patch follow-up on top of v3 that does\nthis, which I'll send separately.\n\nThanks,\nTaylor\n"},{"id":"548070","messageId":"alW0KzSZuZnHmOZD@com-79390","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"Re: [PATCH v3 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-07-14T03:59:39Z","receivedAt":"2026-07-14T03:59:43Z","isPatch":true,"body":"On Mon, Jul 13, 2026 at 04:41:24PM +0200, Patrick Steinhardt wrote:\n> Range-diff versus v2:\n>\n>  1:  baf2adb012 =  1:  7c0dc1be0d odb/source-packed: improve lookup when enumerating objects\n>  2:  57eecf3031 =  2:  2e5908c9c3 pack-bitmap: mark object filter as `const`\n>  -:  ---------- >  3:  f4d66ccfc6 pack-objects: drop unused return value from add_object_entry()\n>  3:  92dd6a6f6e =  4:  af475654b8 pack-bitmap: allow aborting iteration of bitmapped objects\n>  4:  92fe41577d =  5:  6ca42587c9 pack-bitmap: iterate object sources when opening bitmaps\n>  5:  e5d59959e3 =  6:  f62c3bbc81 pack-bitmap: drop `_1` suffix from functions that open bitmaps\n>  6:  ab3547ac2b =  7:  b2d25b6e9b pack-bitmap: introduce function to open bitmap for a single source\n>  7:  026f21f522 =  8:  a5bf309bec odb: introduce object filters to `odb_for_each_object()`\n>  8:  534b25c817 =  9:  600b15a907 builtin/cat-file: filter objects via object database\n\nThanks, this version looks good to me.\n\nThanks,\nTaylor\n"},{"id":"548077","messageId":"alXKvlzBASmRDtQr@pks.im","threadId":"65959","inReplyTo":"alWx1wj1bc48g11X@com-79390","subject":"Re: [PATCH v2 1/8] odb/source-packed: improve lookup when enumerating objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-14T05:35:58Z","receivedAt":"2026-07-14T05:36:06Z","isPatch":true,"body":"On Mon, Jul 13, 2026 at 08:49:43PM -0700, Taylor Blau wrote:\n> On Mon, Jul 13, 2026 at 11:54:43AM +0200, Patrick Steinhardt wrote:\n> > On Fri, Jul 10, 2026 at 03:25:10PM -0700, Taylor Blau wrote:\n> > > On Fri, Jul 10, 2026 at 10:48:53AM +0200, Patrick Steinhardt wrote:\n> > > > Fix the issue by using `packed_object_info()` directly.\n> > >\n> > > What you wrote here makes sense to me insofar as I understand the\n> > > pluggable ODB code.\n> > >\n> > > However, I am confused by the way this function is written in general.\n> > > We use `bsearch_one_midx()` to locate the first possible MIDX position\n> > > in which an object matching the given prefix may exist, which is\n> > > sensible. However, we go from that position up to \"num\", where \"num\" is\n> > > the total number of objects in the MIDX!\n> > >\n> > > Functionally this is not incorrect as we will happily discard objects\n> > > that do not match the prefix. But it causes us to waste CPU cycles\n> > > repeatedly calling `match_hash()` (at least for the first byte of the\n> > > prefix) for objects that we know will match.\n> >\n> > That's not quite true though, as we abort iteration as soon as\n> > `match_hash()` tells us that the prefix doesn't match anymore.\n> \n> Right, we neither iterate through more objects than necessary once we\n> know that `match_hash()` will stop returning true, nor do we emit\n> objects that don't actually match the prefix.\n> \n> What I was trying to say above is that in the special case where our\n> prefix is a single byte long, we don't have to call `match_hash()` at\n> *all*, since we can enumerate just the portion of the fanout for that\n> specific byte, and we know that all such entries will match.\n\nOh, now that's what you're getting at. I don't think that this case ever\nhappens at all right now. I think the shortest prefix that we're ever\nusing should be at least 2 bytes, as we don't treat anything shorter\nthan 4 hex characters as an abbreviated object ID.\n\nThanks for clarifying!\n\nPatrick\n"},{"id":"548078","messageId":"alXKzb-GHodV6uGj@pks.im","threadId":"65959","inReplyTo":"alW0KzSZuZnHmOZD@com-79390","subject":"Re: [PATCH v3 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-14T05:36:13Z","receivedAt":"2026-07-14T05:36:21Z","isPatch":true,"body":"On Mon, Jul 13, 2026 at 08:59:39PM -0700, Taylor Blau wrote:\n> On Mon, Jul 13, 2026 at 04:41:24PM +0200, Patrick Steinhardt wrote:\n> > Range-diff versus v2:\n> >\n> >  1:  baf2adb012 =  1:  7c0dc1be0d odb/source-packed: improve lookup when enumerating objects\n> >  2:  57eecf3031 =  2:  2e5908c9c3 pack-bitmap: mark object filter as `const`\n> >  -:  ---------- >  3:  f4d66ccfc6 pack-objects: drop unused return value from add_object_entry()\n> >  3:  92dd6a6f6e =  4:  af475654b8 pack-bitmap: allow aborting iteration of bitmapped objects\n> >  4:  92fe41577d =  5:  6ca42587c9 pack-bitmap: iterate object sources when opening bitmaps\n> >  5:  e5d59959e3 =  6:  f62c3bbc81 pack-bitmap: drop `_1` suffix from functions that open bitmaps\n> >  6:  ab3547ac2b =  7:  b2d25b6e9b pack-bitmap: introduce function to open bitmap for a single source\n> >  7:  026f21f522 =  8:  a5bf309bec odb: introduce object filters to `odb_for_each_object()`\n> >  8:  534b25c817 =  9:  600b15a907 builtin/cat-file: filter objects via object database\n> \n> Thanks, this version looks good to me.\n\nThanks for your review!\n\nPatrick\n"},{"id":"548086","messageId":"20260714071733.GA4058320@coredump.intra.peff.net","threadId":"65959","inReplyTo":"alWz_gRs_D0Y0aOy@com-79390","subject":"Re: [PATCH v2 3/8] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-07-14T07:17:33Z","receivedAt":"2026-07-14T07:17:34Z","isPatch":true,"body":"On Mon, Jul 13, 2026 at 08:58:54PM -0700, Taylor Blau wrote:\n\n> > That's fair. But adapting `traverse_commit_list()` requires tons of\n> > changes all over the tree, so I'm inclined to rather leave both\n> > `traverse_bitmap_commit_list()` and `traverse_commit_list()` as-is.\n> > Does that work for both of you?\n> \n> I think that it's fine to leave it as-is for the purpose of this series,\n> though I would like to address it.\n\nMe too.\n\n> I don't think we need to adapt `traverse_commit_list()`, though. We can\n> go in the other direction Peff suggested, which would be to split the\n> callback type used by `for_each_bitmapped_object()` from\n> `show_reachable_fn`, keep the former abortable, and make the latter\n> return void.\n> \n> That keeps `traverse_bitmap_commit_list()` in sync with\n> `traverse_commit_list()` without changing the non-bitmap traversal\n> machinery. I have a small two-patch follow-up on top of v3 that does\n> this, which I'll send separately.\n\nThat would be a nice cleanup if it's possible, but I wondered if you\nwould find that one or more of the callbacks actually rely on this abort\nfeature. Only one way to find out. :)\n\n-Peff\n"},{"id":"548087","messageId":"20260714071745.GB4058320@coredump.intra.peff.net","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im","subject":"Re: [PATCH v3 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2026-07-14T07:17:45Z","receivedAt":"2026-07-14T07:17:47Z","isPatch":true,"body":"On Mon, Jul 13, 2026 at 04:41:24PM +0200, Patrick Steinhardt wrote:\n\n> Changes in v3:\n>   - Weave Peff's patch into the patch series.\n>   - Link to v2: https://patch.msgid.link/20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im\n\nYay, thank you. :)\n\n-Peff\n"},{"id":"548205","messageId":"87fr1kn8jr.fsf@emacs.iotcl.com","threadId":"65959","inReplyTo":"alCakwxxOc4FEEAv@pks.im","subject":"Re: [PATCH 6/7] odb: introduce object filters to `odb_for_each_object()`","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-07-15T05:09:12Z","receivedAt":"2026-07-15T05:09:24Z","isPatch":true,"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> An alternative going forward could be to perform filtering of yielded\n> objects inside `odb_for_each_object()` itself so that it will filter out\n> any objects that the backends themselves couldn't filter efficiently.\n> But I'm not sure I want to go there as part of this series -- we only\n> have a single caller anyway that iterates with a filter, and that caller\n> already knows to manually filter references.\n\nAh, that's a valid point. I didn't think of that.\n\nIf only we were using a language that has lazy iterators ;)\n\n-- \nCheers,\nToon\n"},{"id":"548206","messageId":"875x2gn7rx.fsf@emacs.iotcl.com","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-7-b3c65c641073@pks.im","subject":"Re: [PATCH v3 7/9] pack-bitmap: introduce function to open bitmap for a single source","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-07-15T05:25:54Z","receivedAt":"2026-07-15T05:26:00Z","isPatch":true,"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The function `prepare_bitmap_git()` opens the first bitmap it can find\n> in any of the object sources connected to the repository. In a\n> subsequent commit, the \"packed\" object database backend will learn to\n> use bitmaps to answer object filters when enumerating objects. That\n> backend operates on a single object source though, so using a bitmap\n> that potentially belongs to a different source would be wrong:\n>\n>   - The source would yield objects that are not part of the source\n>     itself.\n>\n>   - The object source info would be attributed to the wrong source.\n>\n>   - With multiple sources, each source would enumerate the same bitmap\n>     another time.\n>\n> Introduce a new function `prepare_source_bitmap_git()` that only opens\n> bitmaps belonging to the given object source.\n\nTinies nit: this should be `prepare_bitmap_git_for_source()`\n\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  pack-bitmap.c | 12 ++++++++++++\n>  pack-bitmap.h |  2 ++\n>  2 files changed, 14 insertions(+)\n>\n> diff --git a/pack-bitmap.c b/pack-bitmap.c\n> index 72c8ae3228..09ba15d26b 100644\n> --- a/pack-bitmap.c\n> +++ b/pack-bitmap.c\n> @@ -753,6 +753,18 @@ struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx)\n>  \treturn NULL;\n>  }\n>  \n> +struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source)\n> +{\n> +\tstruct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git));\n> +\n> +\tif (!open_bitmap_for_source(source, bitmap_git) &&\n> +\t    !load_bitmap(source->base.odb->repo, bitmap_git, 0))\n> +\t\treturn bitmap_git;\n> +\n> +\tfree_bitmap_index(bitmap_git);\n> +\treturn NULL;\n> +}\n> +\n>  int bitmap_index_contains_pack(struct bitmap_index *bitmap, struct packed_git *pack)\n>  {\n>  \tfor (; bitmap; bitmap = bitmap->base) {\n> diff --git a/pack-bitmap.h b/pack-bitmap.h\n> index ae8dc491ac..9f20fb6e56 100644\n> --- a/pack-bitmap.h\n> +++ b/pack-bitmap.h\n> @@ -9,6 +9,7 @@\n>  #include \"string-list.h\"\n>  \n>  struct commit;\n> +struct odb_source_packed;\n>  struct repository;\n>  struct rev_info;\n>  \n> @@ -68,6 +69,7 @@ struct bitmapped_pack {\n>  \n>  struct bitmap_index *prepare_bitmap_git(struct repository *r);\n>  struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx);\n> +struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source);\n>  \n>  /*\n>   * Given a bitmap index, determine whether it contains the pack either directly\n>\n> -- \n> 2.55.0.313.g8d093f411d.dirty\n>\n>\n\n-- \nCheers,\nToon\n"},{"id":"548207","messageId":"874ii0n7pv.fsf@emacs.iotcl.com","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-8-b3c65c641073@pks.im","subject":"Re: [PATCH v3 8/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-07-15T05:27:08Z","receivedAt":"2026-07-15T05:27:16Z","isPatch":true,"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> The function `for_each_bitmapped_object()` can be used to iterate\n> through all objects covered by a bitmap. The benefit of this function is\n> that it allows the caller to efficiently handle some object filters. For\n> example, this can be used to filter out objects of a specific type with\n> some simple bitmap operations. But callers are currently required to\n> manually wire up the use of bitmaps though, and to do so they have to\n> reach into internals of a given object database source.\n>\n> Introduce a new `struct odb_for_each_object_options::filter` field so\n> that the interface becomes generic. When set, then a backend may\n> optionally use the filter to skip some objects that it would have\n> otherwise yielded.\n>\n> Note that the respective backends are free to ignore this field if they\n> cannot meaningfully optimize for a given filter, and consequently\n> callers need to verify whether they actually want the returned objects.\n> While annoying, we cannot easily lift this restriction anyway as the\n> object filter infrastructure supports some filters that cannot be\n> answered by the object database alone.\n>\n> An alternative might be to limit the filters to only those that _can_ be\n> answered by backends. But ultimately, the filters that can be answered\n> efficiently by the \"packed\" backend are completely disjunct from those\n> that can be answered by the \"loose\" backend, and consequently the set of\n> filters supported by all backends would be empty. Furthermore, it would\n> require us to make assumptions about capabilities of future backends,\n> which may be able to efficiently handle more filters than current ones.\n> So in the end, this alternative would only limit us artificially.\n>\n> Implement the logic for the \"packed\" source. Note that we use the new\n> function `prepare_source_bitmap_git()` to open the bitmap: as the\n\nAlso here: `prepare_bitmap_git_for_source()`\n\n> backend operates on a single object source, we must only use bitmaps\n> that belong to that specific source. Otherwise we might yield objects\n> that are not part of the source at all, and with multiple sources we\n> would enumerate the same bitmap once per source.\n\n\n-- \nCheers,\nToon\n"},{"id":"548208","messageId":"871pd4n71u.fsf@emacs.iotcl.com","threadId":"65959","inReplyTo":"20260713-pks-odb-for-each-object-filter-v3-9-b3c65c641073@pks.im","subject":"Re: [PATCH v3 9/9] builtin/cat-file: filter objects via object database","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-07-15T05:41:33Z","receivedAt":"2026-07-15T05:41:42Z","isPatch":true,"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> When batching all objects, git-cat-file(1) reaches into the internals of\n> the object database and manually manages bitmaps to apply object\n> filters. This creates coupling between the command and the internals of\n> the respective backend.\n>\n> Refactor git-cat-file(1) to use the new object filter option when\n> batching all objects. This significantly simplifies the logic and\n> ensures that we don't have to reach into internals of the \"files\" source\n> anymore.\n>\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  builtin/cat-file.c | 76 +++++-------------------------------------------------\n>  1 file changed, 7 insertions(+), 69 deletions(-)\n>\n> diff --git a/builtin/cat-file.c b/builtin/cat-file.c\n> index b4b99a73da..1458dd76d6 100644\n> --- a/builtin/cat-file.c\n> +++ b/builtin/cat-file.c\n> @@ -20,7 +20,6 @@\n>  #include \"userdiff.h\"\n>  #include \"oid-array.h\"\n>  #include \"packfile.h\"\n> -#include \"pack-bitmap.h\"\n>  #include \"object-file.h\"\n>  #include \"object-name.h\"\n>  #include \"odb.h\"\n> @@ -844,28 +843,6 @@ static int batch_one_object_oi(const struct object_id *oid,\n>  \treturn payload->callback(oid, NULL, 0, payload->payload);\n>  }\n>  \n> -static int batch_one_object_packed(const struct object_id *oid,\n> -\t\t\t\t   struct packed_git *pack,\n> -\t\t\t\t   uint32_t pos,\n> -\t\t\t\t   void *_payload)\n> -{\n> -\tstruct for_each_object_payload *payload = _payload;\n> -\treturn payload->callback(oid, pack, nth_packed_object_offset(pack, pos),\n> -\t\t\t\t payload->payload);\n> -}\n> -\n> -static int batch_one_object_bitmapped(const struct object_id *oid,\n> -\t\t\t\t      enum object_type type UNUSED,\n> -\t\t\t\t      int flags UNUSED,\n> -\t\t\t\t      uint32_t hash UNUSED,\n> -\t\t\t\t      struct packed_git *pack,\n> -\t\t\t\t      off_t offset,\n> -\t\t\t\t      void *_payload)\n> -{\n> -\tstruct for_each_object_payload *payload = _payload;\n> -\treturn payload->callback(oid, pack, offset, payload->payload);\n> -}\n> -\n>  static void batch_each_object(struct batch_options *opt,\n>  \t\t\t      for_each_object_fn callback,\n>  \t\t\t      unsigned flags,\n> @@ -875,56 +852,17 @@ static void batch_each_object(struct batch_options *opt,\n>  \t\t.callback = callback,\n>  \t\t.payload = _payload,\n>  \t};\n> +\tstruct odb_source_info source_info;\n> +\tstruct object_info oi = {\n> +\t\t.source_infop = &source_info,\n> +\t};\n>  \tstruct odb_for_each_object_options opts = {\n>  \t\t.flags = flags,\n> +\t\t.filter = &opt->objects_filter,\n\nAhha, so here we pass the filter down.\n\n>  \t};\n> -\tstruct bitmap_index *bitmap = NULL;\n> -\tstruct odb_source *source;\n> -\n> -\t/*\n> -\t * TODO: we still need to tap into implementation details of the object\n> -\t * database sources. Ideally, we should extend `odb_for_each_object()`\n> -\t * to handle object filters itself so that we can move the filtering\n> -\t * logic into the individual sources.\n> -\t */\n> -\todb_prepare_alternates(the_repository->objects);\n> -\tfor (source = the_repository->objects->sources; source; source = source->next) {\n> -\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n> -\t\tint ret = odb_source_for_each_object(&files->loose->base, NULL, batch_one_object_oi,\n> -\t\t\t\t\t\t     &payload, &opts);\n> -\t\tif (ret)\n> -\t\t\tbreak;\n> -\t}\n> -\n> -\tif (opt->objects_filter.choice != LOFC_DISABLED &&\n> -\t    (bitmap = prepare_bitmap_git(the_repository)) &&\n> -\t    !for_each_bitmapped_object(bitmap, &opt->objects_filter,\n> -\t\t\t\t       batch_one_object_bitmapped, &payload)) {\n\nSimilar to the new code, the filter is used if possible. It is\nbatch_object_write() which ensures the filter is applied. That didn't\nchange. \n\n> -\t\tstruct packed_git *pack;\n> -\n> -\t\trepo_for_each_pack(the_repository, pack) {\n> -\t\t\tif (bitmap_index_contains_pack(bitmap, pack) ||\n> -\t\t\t    open_pack_index(pack))\n> -\t\t\t\tcontinue;\n> -\t\t\tfor_each_object_in_pack(pack, batch_one_object_packed,\n> -\t\t\t\t\t\t&payload, flags);\n> -\t\t}\n> -\t} else {\n> -\t\tstruct odb_source_info source_info;\n> -\t\tstruct object_info oi = {\n> -\t\t\t.source_infop = &source_info,\n> -\t\t};\n> -\n> -\t\tfor (source = the_repository->objects->sources; source; source = source->next) {\n> -\t\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n> -\t\t\tint ret = odb_source_for_each_object(&files->packed->base, &oi,\n> -\t\t\t\t\t\t\t     batch_one_object_oi, &payload, &opts);\n> -\t\t\tif (ret)\n> -\t\t\t\tbreak;\n> -\t\t}\n> -\t}\n>  \n> -\tfree_bitmap_index(bitmap);\n> +\todb_for_each_object_ext(the_repository->objects, &oi,\n> +\t\t\t\tbatch_one_object_oi, &payload, &opts);\n\nNice to see it abstracted out like this!\n\n>  }\n>  \n>  static int batch_objects(struct batch_options *opt)\n>\n> -- \n> 2.55.0.313.g8d093f411d.dirty\n>\n>\n\n-- \nCheers,\nToon\n"},{"id":"548209","messageId":"87wluwlsdf.fsf@emacs.iotcl.com","threadId":"65959","inReplyTo":"alXKzb-GHodV6uGj@pks.im","subject":"Re: [PATCH v3 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-07-15T05:43:56Z","receivedAt":"2026-07-15T05:44:02Z","isPatch":true,"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> On Mon, Jul 13, 2026 at 08:59:39PM -0700, Taylor Blau wrote:\n>> On Mon, Jul 13, 2026 at 04:41:24PM +0200, Patrick Steinhardt wrote:\n>> > Range-diff versus v2:\n>> >\n>> >  1:  baf2adb012 =  1:  7c0dc1be0d odb/source-packed: improve lookup when enumerating objects\n>> >  2:  57eecf3031 =  2:  2e5908c9c3 pack-bitmap: mark object filter as `const`\n>> >  -:  ---------- >  3:  f4d66ccfc6 pack-objects: drop unused return value from add_object_entry()\n>> >  3:  92dd6a6f6e =  4:  af475654b8 pack-bitmap: allow aborting iteration of bitmapped objects\n>> >  4:  92fe41577d =  5:  6ca42587c9 pack-bitmap: iterate object sources when opening bitmaps\n>> >  5:  e5d59959e3 =  6:  f62c3bbc81 pack-bitmap: drop `_1` suffix from functions that open bitmaps\n>> >  6:  ab3547ac2b =  7:  b2d25b6e9b pack-bitmap: introduce function to open bitmap for a single source\n>> >  7:  026f21f522 =  8:  a5bf309bec odb: introduce object filters to `odb_for_each_object()`\n>> >  8:  534b25c817 =  9:  600b15a907 builtin/cat-file: filter objects via object database\n>> \n>> Thanks, this version looks good to me.\n\nI only posted a nit about function name in the commit messages 7 & 8\nisn't correct, but other than that I'm happy with these patches too.\n\n-- \nCheers,\nToon\n"},{"id":"548212","messageId":"alcnBXTLa7cqrL-d@pks.im","threadId":"65959","inReplyTo":"87wluwlsdf.fsf@emacs.iotcl.com","subject":"Re: [PATCH v3 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:21:57Z","receivedAt":"2026-07-15T06:22:05Z","isPatch":true,"body":"On Wed, Jul 15, 2026 at 07:43:56AM +0200, Toon Claes wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > On Mon, Jul 13, 2026 at 08:59:39PM -0700, Taylor Blau wrote:\n> >> On Mon, Jul 13, 2026 at 04:41:24PM +0200, Patrick Steinhardt wrote:\n> >> > Range-diff versus v2:\n> >> >\n> >> >  1:  baf2adb012 =  1:  7c0dc1be0d odb/source-packed: improve lookup when enumerating objects\n> >> >  2:  57eecf3031 =  2:  2e5908c9c3 pack-bitmap: mark object filter as `const`\n> >> >  -:  ---------- >  3:  f4d66ccfc6 pack-objects: drop unused return value from add_object_entry()\n> >> >  3:  92dd6a6f6e =  4:  af475654b8 pack-bitmap: allow aborting iteration of bitmapped objects\n> >> >  4:  92fe41577d =  5:  6ca42587c9 pack-bitmap: iterate object sources when opening bitmaps\n> >> >  5:  e5d59959e3 =  6:  f62c3bbc81 pack-bitmap: drop `_1` suffix from functions that open bitmaps\n> >> >  6:  ab3547ac2b =  7:  b2d25b6e9b pack-bitmap: introduce function to open bitmap for a single source\n> >> >  7:  026f21f522 =  8:  a5bf309bec odb: introduce object filters to `odb_for_each_object()`\n> >> >  8:  534b25c817 =  9:  600b15a907 builtin/cat-file: filter objects via object database\n> >> \n> >> Thanks, this version looks good to me.\n> \n> I only posted a nit about function name in the commit messages 7 & 8\n> isn't correct, but other than that I'm happy with these patches too.\n\nThanks for your review! I'll send one more iteration that fixes these\nnits.\n\nPatrick\n"},{"id":"548213","messageId":"alcnFcpxNWa150U2@pks.im","threadId":"65959","inReplyTo":"875x2gn7rx.fsf@emacs.iotcl.com","subject":"Re: [PATCH v3 7/9] pack-bitmap: introduce function to open bitmap for a single source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:13Z","receivedAt":"2026-07-15T06:22:19Z","isPatch":true,"body":"On Wed, Jul 15, 2026 at 07:25:54AM +0200, Toon Claes wrote:\n> Patrick Steinhardt <ps@pks.im> writes:\n> \n> > The function `prepare_bitmap_git()` opens the first bitmap it can find\n> > in any of the object sources connected to the repository. In a\n> > subsequent commit, the \"packed\" object database backend will learn to\n> > use bitmaps to answer object filters when enumerating objects. That\n> > backend operates on a single object source though, so using a bitmap\n> > that potentially belongs to a different source would be wrong:\n> >\n> >   - The source would yield objects that are not part of the source\n> >     itself.\n> >\n> >   - The object source info would be attributed to the wrong source.\n> >\n> >   - With multiple sources, each source would enumerate the same bitmap\n> >     another time.\n> >\n> > Introduce a new function `prepare_source_bitmap_git()` that only opens\n> > bitmaps belonging to the given object source.\n> \n> Tinies nit: this should be `prepare_bitmap_git_for_source()`\n\nIndeed it should be, good catch. Shows that I've been iterating on the\nname a bit :)\n\nPatrick\n"},{"id":"548214","messageId":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im","subject":"[PATCH v4 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:30Z","receivedAt":"2026-07-15T06:22:40Z","isPatch":true,"body":"Hi,\n\nthis patch series introduces object filters to `odb_for_each_object()`.\nThe intent of this is to make `git cat-file --batch-all-objects` work\nwith pluggable object databases. Right now it doesn't because it reaches\ninto internals of the \"packed\" backend to efficiently handle bitmapped\nobjects.\n\nThe series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n2026-07-06) with ps/odb-drop-whence at 8a7ad23e11 (odb: document object\ninfo fields, 2026-07-02) merged into it.\n\nChanges in v4:\n  - Fix references to an old function name in commit messages.\n  - Link to v3: https://patch.msgid.link/20260713-pks-odb-for-each-object-filter-v3-0-b3c65c641073@pks.im\n\nChanges in v3:\n  - Weave Peff's patch into the patch series.\n  - Link to v2: https://patch.msgid.link/20260710-pks-odb-for-each-object-filter-v2-0-3710a9cc165a@pks.im\n\nChanges in v2:\n  - Add another patch to drop the `_1()` prefixes that aren't required\n    anymore.\n  - Change the approach in `open_bitmap_for_source()` to also use a\n    `found` boolean instead of a confusing integer.\n  - Add some more explanations to commit messages.\n  - Link to v1: https://patch.msgid.link/20260709-pks-odb-for-each-object-filter-v1-0-82fe014b12b3@pks.im\n\nThanks!\n\nPatrick\n\n---\nJeff King (1):\n      pack-objects: drop unused return value from add_object_entry()\n\nPatrick Steinhardt (8):\n      odb/source-packed: improve lookup when enumerating objects\n      pack-bitmap: mark object filter as `const`\n      pack-bitmap: allow aborting iteration of bitmapped objects\n      pack-bitmap: iterate object sources when opening bitmaps\n      pack-bitmap: drop `_1` suffix from functions that open bitmaps\n      pack-bitmap: introduce function to open bitmap for a single source\n      odb: introduce object filters to `odb_for_each_object()`\n      builtin/cat-file: filter objects via object database\n\n builtin/cat-file.c     |  76 +++--------------------------\n builtin/pack-objects.c |  11 ++---\n builtin/rev-list.c     |   2 +-\n odb.h                  |  12 +++++\n odb/source-packed.c    |  77 ++++++++++++++++++++++++++---\n pack-bitmap.c          | 129 +++++++++++++++++++++++++++----------------------\n pack-bitmap.h          |  10 +++-\n 7 files changed, 175 insertions(+), 142 deletions(-)\n\nRange-diff versus v3:\n\n 1:  2d4bc229ac =  1:  30cf75ce12 odb/source-packed: improve lookup when enumerating objects\n 2:  dfa942f6da =  2:  735eb8ad5c pack-bitmap: mark object filter as `const`\n 3:  9bfba42b27 =  3:  a1f32c5ea6 pack-objects: drop unused return value from add_object_entry()\n 4:  405c77bad3 =  4:  acb0aad580 pack-bitmap: allow aborting iteration of bitmapped objects\n 5:  70e68f10aa =  5:  199db72f4c pack-bitmap: iterate object sources when opening bitmaps\n 6:  08ba2c1db3 =  6:  69d5882b57 pack-bitmap: drop `_1` suffix from functions that open bitmaps\n 7:  d410cfa4e2 !  7:  a1eabafebd pack-bitmap: introduce function to open bitmap for a single source\n    @@ Commit message\n           - With multiple sources, each source would enumerate the same bitmap\n             another time.\n     \n    -    Introduce a new function `prepare_source_bitmap_git()` that only opens\n    -    bitmaps belonging to the given object source.\n    +    Introduce a new function `prepare_bitmap_git_for_source()` that only\n    +    opens bitmaps belonging to the given object source.\n     \n         Signed-off-by: Patrick Steinhardt <ps@pks.im>\n     \n 8:  e0df97e318 !  8:  6ef9885006 odb: introduce object filters to `odb_for_each_object()`\n    @@ Commit message\n         So in the end, this alternative would only limit us artificially.\n     \n         Implement the logic for the \"packed\" source. Note that we use the new\n    -    function `prepare_source_bitmap_git()` to open the bitmap: as the\n    +    function `prepare_bitmap_git_for_source()` to open the bitmap: as the\n         backend operates on a single object source, we must only use bitmaps\n         that belong to that specific source. Otherwise we might yield objects\n         that are not part of the source at all, and with multiple sources we\n 9:  e866b1f4b8 =  9:  104da65906 builtin/cat-file: filter objects via object database\n\n---\nbase-commit: 3c8e2790f2ce15e8b5d4b4e6ced711b12649f32a\nchange-id: 20260708-pks-odb-for-each-object-filter-13286fa3523d\n\n"},{"id":"548215","messageId":"20260715-pks-odb-for-each-object-filter-v4-1-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 1/9] odb/source-packed: improve lookup when enumerating objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:31Z","receivedAt":"2026-07-15T06:22:42Z","isPatch":true,"body":"When iterating through objects of a packed source that have a specific\nprefix we do so via two different methods:\n\n  - When a multi-pack index is available we use that one to efficiently\n    loop through all objects.\n\n  - We then loop through all packfiles that aren't covered by a\n    multi-pack index.\n\nRegardless of which mechanism we use, we then iterate through all the\nobjects indexed by the respective data structure. Curiously though,\nwhile we use the indices for enumerating the objects, we completely\nignore it for the actual object lookup. Instead, we call into the\ngeneric `odb_source_read_object_info()` function, which will itself\nconsult the indices to figure out where the object in question even\nlives.\n\nThis has two consequences:\n\n  - It's inefficient, as we basically have to figure out the position of\n    the object a second time.\n\n  - It's subtly wrong, as it may now happen that a specific object will\n    be looked up via a different pack in case it exists multiple times.\n    This is unlikely to have any real-world consequences, but it's still\n    the wrong thing to do.\n\nFix the issue by using `packed_object_info()` directly. While at it,\nrename the `store` variable to `source`.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n odb/source-packed.c | 15 ++++++++-------\n 1 file changed, 8 insertions(+), 7 deletions(-)\n\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 0edea5356d..9cfa02b7a2 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -143,7 +143,7 @@ static bool should_exclude_pack(struct packed_git *p, enum odb_for_each_object_f\n }\n \n static int for_each_prefixed_object_in_midx(\n-\tstruct odb_source_packed *store,\n+\tstruct odb_source_packed *source,\n \tstruct multi_pack_index *m,\n \tconst struct odb_for_each_object_options *opts,\n \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n@@ -170,6 +170,7 @@ static int for_each_prefixed_object_in_midx(\n \t\t */\n \t\tfor (i = first; i < num; i++) {\n \t\t\tconst struct object_id *current = NULL;\n+\t\t\tstruct packed_git *pack;\n \t\t\tstruct object_id oid;\n \n \t\t\tcurrent = nth_midxed_object_oid(&oid, m, i);\n@@ -177,9 +178,8 @@ static int for_each_prefixed_object_in_midx(\n \t\t\tif (!match_hash(len, opts->prefix->hash, current->hash))\n \t\t\t\tbreak;\n \n-\t\t\tif (opts->flags) {\n+\t\t\tif (opts->flags || data->request) {\n \t\t\t\tuint32_t pack_id = nth_midxed_pack_int_id(m, i);\n-\t\t\t\tstruct packed_git *pack;\n \n \t\t\t\tif (prepare_midx_pack(m, pack_id)) {\n \t\t\t\t\tpack_errors = true;\n@@ -193,9 +193,9 @@ static int for_each_prefixed_object_in_midx(\n \n \t\t\tif (data->request) {\n \t\t\t\tstruct object_info oi = *data->request;\n+\t\t\t\toff_t offset = nth_midxed_offset(m, i);\n \n-\t\t\t\tret = odb_source_read_object_info(&store->base, current,\n-\t\t\t\t\t\t\t\t  &oi, 0);\n+\t\t\t\tret = packed_object_info(source, pack, offset, &oi);\n \t\t\t\tif (ret)\n \t\t\t\t\tgoto out;\n \n@@ -219,7 +219,7 @@ static int for_each_prefixed_object_in_midx(\n }\n \n static int for_each_prefixed_object_in_pack(\n-\tstruct odb_source_packed *store,\n+\tstruct odb_source_packed *source,\n \tstruct packed_git *p,\n \tconst struct odb_for_each_object_options *opts,\n \tstruct odb_source_packed_for_each_object_wrapper_data *data)\n@@ -246,8 +246,9 @@ static int for_each_prefixed_object_in_pack(\n \n \t\tif (data->request) {\n \t\t\tstruct object_info oi = *data->request;\n+\t\t\toff_t offset = nth_packed_object_offset(p, i);\n \n-\t\t\tret = odb_source_read_object_info(&store->base, &oid, &oi, 0);\n+\t\t\tret = packed_object_info(source, p, offset, &oi);\n \t\t\tif (ret)\n \t\t\t\tgoto out;\n \n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548216","messageId":"20260715-pks-odb-for-each-object-filter-v4-2-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 2/9] pack-bitmap: mark object filter as `const`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:32Z","receivedAt":"2026-07-15T06:22:45Z","isPatch":true,"body":"The function `for_each_bitmapped_object()` accepts an optional object\nfilter. This filter is never modified by the function, but is not\ndeclared as `const`. Fix this.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 6 +++---\n pack-bitmap.h | 2 +-\n 2 files changed, 4 insertions(+), 4 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 35774b6f0c..a47c231632 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1976,7 +1976,7 @@ static void filter_bitmap_object_type(struct bitmap_index *bitmap_git,\n static int filter_bitmap(struct bitmap_index *bitmap_git,\n \t\t\t struct object_list *tip_objects,\n \t\t\t struct bitmap *to_filter,\n-\t\t\t struct list_objects_filter_options *filter)\n+\t\t\t const struct list_objects_filter_options *filter)\n {\n \tif (!filter || filter->choice == LOFC_DISABLED)\n \t\treturn 0;\n@@ -2027,7 +2027,7 @@ static int filter_bitmap(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int can_filter_bitmap(struct list_objects_filter_options *filter)\n+static int can_filter_bitmap(const struct list_objects_filter_options *filter)\n {\n \treturn !filter_bitmap(NULL, NULL, NULL, filter);\n }\n@@ -2058,7 +2058,7 @@ static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,\n }\n \n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n-\t\t\t      struct list_objects_filter_options *filter,\n+\t\t\t      const struct list_objects_filter_options *filter,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload)\n {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 19a8655457..47935eb24e 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -96,7 +96,7 @@ struct list_objects_filter_options;\n  * not supported, `0` otherwise.\n  */\n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n-\t\t\t      struct list_objects_filter_options *filter,\n+\t\t\t      const struct list_objects_filter_options *filter,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload);\n \n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548217","messageId":"20260715-pks-odb-for-each-object-filter-v4-3-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 3/9] pack-objects: drop unused return value from add_object_entry()","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:33Z","receivedAt":"2026-07-15T06:22:47Z","isPatch":true,"body":"From: Jeff King <peff@peff.net>\n\nThis function returns 0/1 to its caller to tell them whether we actually\nadded a new entry (or if we considered it redundant). But nobody has\nrelied on that behavior since 5379a5c5ee (Thin pack generation:\noptimization., 2006-04-05).\n\nThe extra return does not hurt much, but it is a bit confusing. We have\na sister function, add_object_entry_from_bitmap(), which has the same\nreturn value semantics. That function is about to change to always return\n0 (not void, because it must conform to a callback function interface).\nSo with that change, we'd have two related functions which both return\nan \"int\" but with different semantics.\n\nLet's drop the unused \"int\" return from add_object_entry() entirely,\nwhich makes it more clear that the two functions have diverged.\n\nSigned-off-by: Jeff King <peff@peff.net>\n[ps: slightly massaged the commit message]\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/pack-objects.c | 9 ++++-----\n 1 file changed, 4 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex ea5eab4cf8..188c4f6d4b 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1867,8 +1867,8 @@ static const char no_closure_warning[] = N_(\n \"disabling bitmap writing, as some objects are not being packed\"\n );\n \n-static int add_object_entry(const struct object_id *oid, enum object_type type,\n-\t\t\t    const char *name, int exclude)\n+static void add_object_entry(const struct object_id *oid, enum object_type type,\n+\t\t\t     const char *name, int exclude)\n {\n \tstruct packed_git *found_pack = NULL;\n \toff_t found_offset = 0;\n@@ -1876,7 +1876,7 @@ static int add_object_entry(const struct object_id *oid, enum object_type type,\n \tdisplay_progress(progress_state, ++nr_seen);\n \n \tif (have_duplicate_entry(oid, exclude))\n-\t\treturn 0;\n+\t\treturn;\n \n \tif (!want_object_in_pack(oid, exclude, &found_pack, &found_offset)) {\n \t\t/* The pack is missing an object, so it will not have closure */\n@@ -1885,13 +1885,12 @@ static int add_object_entry(const struct object_id *oid, enum object_type type,\n \t\t\t\twarning(_(no_closure_warning));\n \t\t\twrite_bitmap_index = 0;\n \t\t}\n-\t\treturn 0;\n+\t\treturn;\n \t}\n \n \tcreate_object_entry(oid, type, pack_name_hash_fn(name),\n \t\t\t    exclude, name && no_try_delta(name),\n \t\t\t    found_pack, found_offset);\n-\treturn 1;\n }\n \n static int add_object_entry_from_bitmap(const struct object_id *oid,\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548218","messageId":"20260715-pks-odb-for-each-object-filter-v4-4-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 4/9] pack-bitmap: allow aborting iteration of bitmapped objects","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:34Z","receivedAt":"2026-07-15T06:22:49Z","isPatch":true,"body":"In a subsequent commit we'll lift iteration of bitmapped objects into\nthe \"packed\" backend and make it accessible via `odb_for_each_object()`.\nThe calling convention for that function is that the callback may return\na non-zero exit code, and if so we'll abort iteration. This is currently\nimpossible to realize though, as `for_each_bitmapped_object()` will\nignore any return value and just churn through all objects completely.\n\nThis doesn't matter to the callers of `for_each_bitmapped_object()`, as\nthere's only one of them in git-cat-file(1), and the callbacks we pass\nalways return zero. But once we move the logic into the generic\ninfrastructure it becomes a latent bug waiting to happen.\n\nRefactor the code so that the return value of the `show_reach` callback\nis not ignored anymore. Instead, returning a non-zero value will cause\nus to abort iteration in both `show_objects_for_type()` and in\n`for_each_bitmapped_object()`.\n\nNote though that there's a second user of `show_objects_for_type()` with\n`traverse_bitmap_commit_list()`, and that function does indeed invoke\ncallbacks that may return non-zero. This non-zero return value never had\nany effect at all though, and the callbacks that return non-zero values\nare only ever invoked via `traverse_bitmap_commit_list()`. Consequently,\nwe adapt them to always return 0.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/pack-objects.c |  2 +-\n builtin/rev-list.c     |  2 +-\n pack-bitmap.c          | 31 +++++++++++++++++++++----------\n pack-bitmap.h          |  3 ++-\n 4 files changed, 25 insertions(+), 13 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 188c4f6d4b..3673b14b89 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1908,7 +1908,7 @@ static int add_object_entry_from_bitmap(const struct object_id *oid,\n \t\treturn 0;\n \n \tcreate_object_entry(oid, type, name_hash, 0, 0, pack, offset);\n-\treturn 1;\n+\treturn 0;\n }\n \n struct pbase_tree_cache {\ndiff --git a/builtin/rev-list.c b/builtin/rev-list.c\nindex 8f63003709..02818b81c6 100644\n--- a/builtin/rev-list.c\n+++ b/builtin/rev-list.c\n@@ -486,7 +486,7 @@ static int show_object_fast(\n \tvoid *payload UNUSED)\n {\n \tfprintf(stdout, \"%s\\n\", oid_to_hex(oid));\n-\treturn 1;\n+\treturn 0;\n }\n \n static void print_disk_usage(off_t size)\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex a47c231632..eda38a5433 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -1695,7 +1695,7 @@ static void init_type_iterator(struct ewah_or_iterator *it,\n \t}\n }\n \n-static void show_objects_for_type(\n+static int show_objects_for_type(\n \tstruct bitmap_index *bitmap_git,\n \tstruct bitmap *objects,\n \tenum object_type object_type,\n@@ -1704,6 +1704,7 @@ static void show_objects_for_type(\n {\n \tsize_t i = 0;\n \tuint32_t offset;\n+\tint ret;\n \n \tstruct ewah_or_iterator it;\n \teword_t filter;\n@@ -1749,11 +1750,17 @@ static void show_objects_for_type(\n \n \t\t\thash = bitmap_name_hash(bitmap_git, index_pos);\n \n-\t\t\tshow_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n+\t\t\tret = show_reach(&oid, object_type, 0, hash, pack, ofs, payload);\n+\t\t\tif (ret)\n+\t\t\t\tgoto out;\n \t\t}\n \t}\n \n+\tret = 0;\n+\n+out:\n \tewah_or_iterator_release(&it);\n+\treturn ret;\n }\n \n static int in_bitmapped_pack(struct bitmap_index *bitmap_git,\n@@ -2062,6 +2069,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\t\t      show_reachable_fn show_reach,\n \t\t\t      void *payload)\n {\n+\tconst enum object_type types[] = {\n+\t\tOBJ_COMMIT,\n+\t\tOBJ_TREE,\n+\t\tOBJ_BLOB,\n+\t\tOBJ_TAG,\n+\t};\n \tstruct bitmap *filtered_bitmap = NULL;\n \tuint32_t objects_nr;\n \tsize_t full_word_count;\n@@ -2086,14 +2099,12 @@ int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\tgoto out;\n \t}\n \n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_COMMIT, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_TREE, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_BLOB, show_reach, payload);\n-\tshow_objects_for_type(bitmap_git, filtered_bitmap,\n-\t\t\t      OBJ_TAG, show_reach, payload);\n+\tfor (size_t i = 0; i < ARRAY_SIZE(types); i++) {\n+\t\tret = show_objects_for_type(bitmap_git, filtered_bitmap,\n+\t\t\t\t\t    types[i], show_reach, payload);\n+\t\tif (ret)\n+\t\t\tgoto out;\n+\t}\n \n \tret = 0;\n out:\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 47935eb24e..ae8dc491ac 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -93,7 +93,8 @@ struct list_objects_filter_options;\n /*\n  * Filter bitmapped objects and iterate through all resulting objects,\n  * executing `show_reach` for each of them. Returns `-1` in case the filter is\n- * not supported, `0` otherwise.\n+ * not supported, `0` otherwise. Aborts iteration and bubbles up the return\n+ * value in case `show_reach()` returns non-zero.\n  */\n int for_each_bitmapped_object(struct bitmap_index *bitmap_git,\n \t\t\t      const struct list_objects_filter_options *filter,\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548219","messageId":"20260715-pks-odb-for-each-object-filter-v4-5-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 5/9] pack-bitmap: iterate object sources when opening bitmaps","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:35Z","receivedAt":"2026-07-15T06:22:52Z","isPatch":true,"body":"When opening a bitmap for a repository we perform two steps:\n\n  - We first look for a multi-pack index bitmap in any of the object\n    sources connected to the repository.\n\n  - We then look for a packfile bitmap in any of the packfiles of any of\n    the object sources.\n\nBoth of these steps thus iterate through object sources themselves, one\nvia `odb_prepare_alternates()` and one via `repo_for_each_pack()`. This\nlayout makes it hard to introduce a way to open the bitmap of one\nspecific object source, which is functionality that we'll require in a\nsubsequent commit.\n\nReverse the loop so that we instead loop through all sources in the\nouter loop, and then for each source we try to load its bitmap via\neither the multi-pack index or via a packfile.\n\nNote that this changes the precedence of bitmaps in one specific edge\ncase: when an earlier object source only has a packfile bitmap, but a\nlater source has a multi-pack index bitmap, we now pick the packfile\nbitmap of the earlier source. Previously, a multi-pack index bitmap from\nany source would have taken precedence over all packfile bitmaps. Given\nthat object sources are ordered such that the local source comes first,\nthis arguably is an improvement, as we now prefer local bitmaps over\nbitmaps in alternates. Furthermore, we already warn about repositories\nthat have multiple bitmaps, so this setup is broken and thus arguably\nnot worth worrying about too much.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 69 +++++++++++++++++++++++++++--------------------------------\n 1 file changed, 31 insertions(+), 38 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex eda38a5433..e32795a595 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -680,60 +680,53 @@ static int load_bitmap(struct repository *r, struct bitmap_index *bitmap_git,\n \treturn 0;\n }\n \n-static int open_pack_bitmap(struct repository *r,\n-\t\t\t    struct bitmap_index *bitmap_git)\n+static int open_bitmap_for_source(struct odb_source_packed *source,\n+\t\t\t\t  struct bitmap_index *bitmap_git)\n {\n-\tstruct packed_git *p;\n-\tint ret = -1;\n+\tstruct multi_pack_index *midx = get_multi_pack_index(source);\n+\tstruct packfile_list_entry *e;\n+\tbool found = false;\n \n-\trepo_for_each_pack(r, p) {\n-\t\tif (open_pack_bitmap_1(bitmap_git, p) == 0) {\n-\t\t\tret = 0;\n-\t\t\t/*\n-\t\t\t * The only reason to keep looking is to report\n-\t\t\t * duplicates.\n-\t\t\t */\n-\t\t\tif (!trace2_is_enabled())\n-\t\t\t\tbreak;\n-\t\t}\n+\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n+\t\tfound = true;\n+\n+\tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n+\t\t/*\n+\t\t * When tracing is enabled we want to keep looking to report\n+\t\t * duplicates even if we have already found a bitmap.\n+\t\t */\n+\t\tif (found && !trace2_is_enabled())\n+\t\t\tbreak;\n+\n+\t\tif (!open_pack_bitmap_1(bitmap_git, e->pack))\n+\t\t\tfound = true;\n \t}\n \n-\treturn ret;\n+\treturn found ? 0 : -1;\n }\n \n-static int open_midx_bitmap(struct repository *r,\n-\t\t\t    struct bitmap_index *bitmap_git)\n+static int open_bitmap(struct repository *r,\n+\t\t       struct bitmap_index *bitmap_git)\n {\n \tstruct odb_source *source;\n-\tint ret = -1;\n+\tbool found = false;\n \n \tassert(!bitmap_git->map);\n \n \todb_prepare_alternates(r->objects);\n \tfor (source = r->objects->sources; source; source = source->next) {\n \t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\tstruct multi_pack_index *midx = get_multi_pack_index(files->packed);\n-\t\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n-\t\t\tret = 0;\n-\t}\n-\treturn ret;\n-}\n-\n-static int open_bitmap(struct repository *r,\n-\t\t       struct bitmap_index *bitmap_git)\n-{\n-\tint found;\n \n-\tassert(!bitmap_git->map);\n+\t\tif (!open_bitmap_for_source(files->packed, bitmap_git))\n+\t\t\tfound = true;\n \n-\tfound = !open_midx_bitmap(r, bitmap_git);\n-\n-\t/*\n-\t * these will all be skipped if we opened a midx bitmap; but run it\n-\t * anyway if tracing is enabled to report the duplicates\n-\t */\n-\tif (!found || trace2_is_enabled())\n-\t\tfound |= !open_pack_bitmap(r, bitmap_git);\n+\t\t/*\n+\t\t * The only reason to keep looking after having found a bitmap\n+\t\t * is to report duplicates.\n+\t\t */\n+\t\tif (found && !trace2_is_enabled())\n+\t\t\tbreak;\n+\t}\n \n \treturn found ? 0 : -1;\n }\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548220","messageId":"20260715-pks-odb-for-each-object-filter-v4-6-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 6/9] pack-bitmap: drop `_1` suffix from functions that open bitmaps","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:36Z","receivedAt":"2026-07-15T06:22:55Z","isPatch":true,"body":"In the preceding commit we've refactored how we open bitmaps. As part of\nthe refactoring we have consolidated `open_pack_bitmap()` as well as\n`open_midx_bitmap()` into `open_bitmap_for_source()`. Consequently, we\nonly have their `open_pack_bitmap_1()` and `open_midx_bitmap_1()`\nvariants left over, where the `_1` suffix doesn't really make much sense\nanymore.\n\nDrop the suffix.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 14 +++++++-------\n 1 file changed, 7 insertions(+), 7 deletions(-)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex e32795a595..72c8ae3228 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -460,8 +460,8 @@ char *pack_bitmap_filename(struct packed_git *p)\n \treturn xstrfmt(\"%.*s.bitmap\", (int)len, p->pack_name);\n }\n \n-static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n-\t\t\t      struct multi_pack_index *midx)\n+static int open_midx_bitmap(struct bitmap_index *bitmap_git,\n+\t\t\t    struct multi_pack_index *midx)\n {\n \tstruct stat st;\n \tchar *bitmap_name = midx_bitmap_filename(midx);\n@@ -539,7 +539,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int open_pack_bitmap_1(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n+static int open_pack_bitmap(struct bitmap_index *bitmap_git, struct packed_git *packfile)\n {\n \tint fd;\n \tstruct stat st;\n@@ -603,7 +603,7 @@ static int load_reverse_index(struct repository *r, struct bitmap_index *bitmap_\n \n \t\t/*\n \t\t * The multi-pack-index's .rev file is already loaded via\n-\t\t * open_pack_bitmap_1().\n+\t\t * open_pack_bitmap().\n \t\t *\n \t\t * But we still need to open the individual pack .rev files,\n \t\t * since we will need to make use of them in pack-objects.\n@@ -687,7 +687,7 @@ static int open_bitmap_for_source(struct odb_source_packed *source,\n \tstruct packfile_list_entry *e;\n \tbool found = false;\n \n-\tif (midx && !open_midx_bitmap_1(bitmap_git, midx))\n+\tif (midx && !open_midx_bitmap(bitmap_git, midx))\n \t\tfound = true;\n \n \tfor (e = packfile_store_get_packs(source); e; e = e->next) {\n@@ -698,7 +698,7 @@ static int open_bitmap_for_source(struct odb_source_packed *source,\n \t\tif (found && !trace2_is_enabled())\n \t\t\tbreak;\n \n-\t\tif (!open_pack_bitmap_1(bitmap_git, e->pack))\n+\t\tif (!open_pack_bitmap(bitmap_git, e->pack))\n \t\t\tfound = true;\n \t}\n \n@@ -746,7 +746,7 @@ struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx)\n {\n \tstruct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git));\n \n-\tif (!open_midx_bitmap_1(bitmap_git, midx))\n+\tif (!open_midx_bitmap(bitmap_git, midx))\n \t\treturn bitmap_git;\n \n \tfree_bitmap_index(bitmap_git);\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548221","messageId":"20260715-pks-odb-for-each-object-filter-v4-7-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 7/9] pack-bitmap: introduce function to open bitmap for a single source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:37Z","receivedAt":"2026-07-15T06:22:57Z","isPatch":true,"body":"The function `prepare_bitmap_git()` opens the first bitmap it can find\nin any of the object sources connected to the repository. In a\nsubsequent commit, the \"packed\" object database backend will learn to\nuse bitmaps to answer object filters when enumerating objects. That\nbackend operates on a single object source though, so using a bitmap\nthat potentially belongs to a different source would be wrong:\n\n  - The source would yield objects that are not part of the source\n    itself.\n\n  - The object source info would be attributed to the wrong source.\n\n  - With multiple sources, each source would enumerate the same bitmap\n    another time.\n\nIntroduce a new function `prepare_bitmap_git_for_source()` that only\nopens bitmaps belonging to the given object source.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n pack-bitmap.c | 12 ++++++++++++\n pack-bitmap.h |  2 ++\n 2 files changed, 14 insertions(+)\n\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 72c8ae3228..09ba15d26b 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -753,6 +753,18 @@ struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx)\n \treturn NULL;\n }\n \n+struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source)\n+{\n+\tstruct bitmap_index *bitmap_git = xcalloc(1, sizeof(*bitmap_git));\n+\n+\tif (!open_bitmap_for_source(source, bitmap_git) &&\n+\t    !load_bitmap(source->base.odb->repo, bitmap_git, 0))\n+\t\treturn bitmap_git;\n+\n+\tfree_bitmap_index(bitmap_git);\n+\treturn NULL;\n+}\n+\n int bitmap_index_contains_pack(struct bitmap_index *bitmap, struct packed_git *pack)\n {\n \tfor (; bitmap; bitmap = bitmap->base) {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex ae8dc491ac..9f20fb6e56 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -9,6 +9,7 @@\n #include \"string-list.h\"\n \n struct commit;\n+struct odb_source_packed;\n struct repository;\n struct rev_info;\n \n@@ -68,6 +69,7 @@ struct bitmapped_pack {\n \n struct bitmap_index *prepare_bitmap_git(struct repository *r);\n struct bitmap_index *prepare_midx_bitmap_git(struct multi_pack_index *midx);\n+struct bitmap_index *prepare_bitmap_git_for_source(struct odb_source_packed *source);\n \n /*\n  * Given a bitmap index, determine whether it contains the pack either directly\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548222","messageId":"20260715-pks-odb-for-each-object-filter-v4-8-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 8/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:38Z","receivedAt":"2026-07-15T06:22:59Z","isPatch":true,"body":"The function `for_each_bitmapped_object()` can be used to iterate\nthrough all objects covered by a bitmap. The benefit of this function is\nthat it allows the caller to efficiently handle some object filters. For\nexample, this can be used to filter out objects of a specific type with\nsome simple bitmap operations. But callers are currently required to\nmanually wire up the use of bitmaps though, and to do so they have to\nreach into internals of a given object database source.\n\nIntroduce a new `struct odb_for_each_object_options::filter` field so\nthat the interface becomes generic. When set, then a backend may\noptionally use the filter to skip some objects that it would have\notherwise yielded.\n\nNote that the respective backends are free to ignore this field if they\ncannot meaningfully optimize for a given filter, and consequently\ncallers need to verify whether they actually want the returned objects.\nWhile annoying, we cannot easily lift this restriction anyway as the\nobject filter infrastructure supports some filters that cannot be\nanswered by the object database alone.\n\nAn alternative might be to limit the filters to only those that _can_ be\nanswered by backends. But ultimately, the filters that can be answered\nefficiently by the \"packed\" backend are completely disjunct from those\nthat can be answered by the \"loose\" backend, and consequently the set of\nfilters supported by all backends would be empty. Furthermore, it would\nrequire us to make assumptions about capabilities of future backends,\nwhich may be able to efficiently handle more filters than current ones.\nSo in the end, this alternative would only limit us artificially.\n\nImplement the logic for the \"packed\" source. Note that we use the new\nfunction `prepare_bitmap_git_for_source()` to open the bitmap: as the\nbackend operates on a single object source, we must only use bitmaps\nthat belong to that specific source. Otherwise we might yield objects\nthat are not part of the source at all, and with multiple sources we\nwould enumerate the same bitmap once per source.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n odb.h               | 12 +++++++++++\n odb/source-packed.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++++\n pack-bitmap.c       |  3 +--\n pack-bitmap.h       |  3 +++\n 4 files changed, 78 insertions(+), 2 deletions(-)\n\ndiff --git a/odb.h b/odb.h\nindex a1e222f605..67d0b34942 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -8,6 +8,7 @@\n #include \"thread-utils.h\"\n \n struct cached_object_entry;\n+struct list_objects_filter_options;\n struct odb_source_inmemory;\n struct packed_git;\n struct repository;\n@@ -490,6 +491,17 @@ struct odb_for_each_object_options {\n \t */\n \tconst struct object_id *prefix;\n \tsize_t prefix_hex_len;\n+\n+\t/*\n+\t * Optional object filter that allows backends to skip yielding\n+\t * objects that are excluded by the filter as an optimization. The\n+\t * filter is a best-effort hint: backends may use it to skip\n+\t * excluded objects (e.g. by consulting a reachability bitmap), but\n+\t * are also free to ignore it entirely and yield every object. As a\n+\t * consequence, callers must re-apply the filter on yielded objects\n+\t * if they require strict filtering semantics.\n+\t */\n+\tconst struct list_objects_filter_options *filter;\n };\n \n /*\ndiff --git a/odb/source-packed.c b/odb/source-packed.c\nindex 9cfa02b7a2..4777395053 100644\n--- a/odb/source-packed.c\n+++ b/odb/source-packed.c\n@@ -3,11 +3,13 @@\n #include \"chdir-notify.h\"\n #include \"dir.h\"\n #include \"git-zlib.h\"\n+#include \"list-objects-filter-options.h\"\n #include \"mergesort.h\"\n #include \"midx.h\"\n #include \"odb/source-packed.h\"\n #include \"odb/streaming.h\"\n #include \"packfile.h\"\n+#include \"pack-bitmap.h\"\n \n static int find_pack_entry(struct odb_source_packed *store,\n \t\t\t   const struct object_id *oid,\n@@ -315,6 +317,37 @@ static int odb_source_packed_for_each_prefixed_object(\n \treturn ret;\n }\n \n+struct bitmapped_for_each_object_data {\n+\tstruct odb_source_packed *packed;\n+\tconst struct object_info *request;\n+\tconst struct odb_for_each_object_options *opts;\n+\todb_for_each_object_cb cb;\n+\tvoid *cb_data;\n+};\n+\n+static int bitmapped_for_each_object(const struct object_id *oid,\n+\t\t\t\t     enum object_type type UNUSED,\n+\t\t\t\t     int flags UNUSED,\n+\t\t\t\t     uint32_t hash UNUSED,\n+\t\t\t\t     struct packed_git *pack,\n+\t\t\t\t     off_t offset,\n+\t\t\t\t     void *cb_data)\n+{\n+\tstruct bitmapped_for_each_object_data *data = cb_data;\n+\n+\tif (should_exclude_pack(pack, data->opts->flags))\n+\t\treturn 0;\n+\n+\tif (data->request) {\n+\t\tstruct object_info oi = *data->request;\n+\t\tif (packed_object_info(data->packed, pack, offset, &oi) < 0)\n+\t\t\treturn -1;\n+\t\treturn data->cb(oid, &oi, data->cb_data);\n+\t}\n+\n+\treturn data->cb(oid, NULL, data->cb_data);\n+}\n+\n static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\t\t\t\t     const struct object_info *request,\n \t\t\t\t\t     odb_for_each_object_cb cb,\n@@ -328,12 +361,33 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\t.cb = cb,\n \t\t.cb_data = cb_data,\n \t};\n+\tstruct bitmap_index *bitmap = NULL;\n \tstruct packfile_list_entry *e;\n \tint pack_errors = 0, ret;\n \n \tif (opts->prefix)\n \t\treturn odb_source_packed_for_each_prefixed_object(packed, opts, &data);\n \n+\tif (opts->filter &&\n+\t    opts->filter->choice != LOFC_DISABLED &&\n+\t    can_filter_bitmap(opts->filter))\n+\t\tbitmap = prepare_bitmap_git_for_source(packed);\n+\tif (bitmap) {\n+\t\tstruct bitmapped_for_each_object_data bitmap_data = {\n+\t\t\t.packed = packed,\n+\t\t\t.request = request,\n+\t\t\t.opts = opts,\n+\t\t\t.cb = cb,\n+\t\t\t.cb_data = cb_data,\n+\t\t};\n+\n+\t\tret = for_each_bitmapped_object(bitmap, opts->filter,\n+\t\t\t\t\t\tbitmapped_for_each_object,\n+\t\t\t\t\t\t&bitmap_data);\n+\t\tif (ret)\n+\t\t\tgoto out;\n+\t}\n+\n \tpacked->skip_mru_updates = true;\n \n \tfor (e = packfile_store_get_packs(packed); e; e = e->next) {\n@@ -342,6 +396,13 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \t\tif (should_exclude_pack(p, opts->flags))\n \t\t\tcontinue;\n \n+\t\t/*\n+\t\t * Objects covered by the bitmap have already been yielded\n+\t\t * above; skip them here to avoid duplicates.\n+\t\t */\n+\t\tif (bitmap && bitmap_index_contains_pack(bitmap, p))\n+\t\t\tcontinue;\n+\n \t\tif (open_pack_index(p)) {\n \t\t\tpack_errors = 1;\n \t\t\tcontinue;\n@@ -357,6 +418,7 @@ static int odb_source_packed_for_each_object(struct odb_source *source,\n \n out:\n \tpacked->skip_mru_updates = false;\n+\tfree_bitmap_index(bitmap);\n \n \tif (!ret && pack_errors)\n \t\tret = -1;\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 09ba15d26b..f55a0859ea 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -2039,12 +2039,11 @@ static int filter_bitmap(struct bitmap_index *bitmap_git,\n \treturn -1;\n }\n \n-static int can_filter_bitmap(const struct list_objects_filter_options *filter)\n+bool can_filter_bitmap(const struct list_objects_filter_options *filter)\n {\n \treturn !filter_bitmap(NULL, NULL, NULL, filter);\n }\n \n-\n static void filter_packed_objects_from_bitmap(struct bitmap_index *bitmap_git,\n \t\t\t\t\t      struct bitmap *result)\n {\ndiff --git a/pack-bitmap.h b/pack-bitmap.h\nindex 9f20fb6e56..1385027c1f 100644\n--- a/pack-bitmap.h\n+++ b/pack-bitmap.h\n@@ -92,6 +92,9 @@ int test_bitmap_pseudo_merge_objects(struct repository *r, uint32_t n);\n \n struct list_objects_filter_options;\n \n+/* Check whether the filter can be computed via the bitmap. */\n+bool can_filter_bitmap(const struct list_objects_filter_options *filter);\n+\n /*\n  * Filter bitmapped objects and iterate through all resulting objects,\n  * executing `show_reach` for each of them. Returns `-1` in case the filter is\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548223","messageId":"20260715-pks-odb-for-each-object-filter-v4-9-616d7adf7fb7@pks.im","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"[PATCH v4 9/9] builtin/cat-file: filter objects via object database","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-07-15T06:22:39Z","receivedAt":"2026-07-15T06:23:02Z","isPatch":true,"body":"When batching all objects, git-cat-file(1) reaches into the internals of\nthe object database and manually manages bitmaps to apply object\nfilters. This creates coupling between the command and the internals of\nthe respective backend.\n\nRefactor git-cat-file(1) to use the new object filter option when\nbatching all objects. This significantly simplifies the logic and\nensures that we don't have to reach into internals of the \"files\" source\nanymore.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/cat-file.c | 76 +++++-------------------------------------------------\n 1 file changed, 7 insertions(+), 69 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex b4b99a73da..1458dd76d6 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -20,7 +20,6 @@\n #include \"userdiff.h\"\n #include \"oid-array.h\"\n #include \"packfile.h\"\n-#include \"pack-bitmap.h\"\n #include \"object-file.h\"\n #include \"object-name.h\"\n #include \"odb.h\"\n@@ -844,28 +843,6 @@ static int batch_one_object_oi(const struct object_id *oid,\n \treturn payload->callback(oid, NULL, 0, payload->payload);\n }\n \n-static int batch_one_object_packed(const struct object_id *oid,\n-\t\t\t\t   struct packed_git *pack,\n-\t\t\t\t   uint32_t pos,\n-\t\t\t\t   void *_payload)\n-{\n-\tstruct for_each_object_payload *payload = _payload;\n-\treturn payload->callback(oid, pack, nth_packed_object_offset(pack, pos),\n-\t\t\t\t payload->payload);\n-}\n-\n-static int batch_one_object_bitmapped(const struct object_id *oid,\n-\t\t\t\t      enum object_type type UNUSED,\n-\t\t\t\t      int flags UNUSED,\n-\t\t\t\t      uint32_t hash UNUSED,\n-\t\t\t\t      struct packed_git *pack,\n-\t\t\t\t      off_t offset,\n-\t\t\t\t      void *_payload)\n-{\n-\tstruct for_each_object_payload *payload = _payload;\n-\treturn payload->callback(oid, pack, offset, payload->payload);\n-}\n-\n static void batch_each_object(struct batch_options *opt,\n \t\t\t      for_each_object_fn callback,\n \t\t\t      unsigned flags,\n@@ -875,56 +852,17 @@ static void batch_each_object(struct batch_options *opt,\n \t\t.callback = callback,\n \t\t.payload = _payload,\n \t};\n+\tstruct odb_source_info source_info;\n+\tstruct object_info oi = {\n+\t\t.source_infop = &source_info,\n+\t};\n \tstruct odb_for_each_object_options opts = {\n \t\t.flags = flags,\n+\t\t.filter = &opt->objects_filter,\n \t};\n-\tstruct bitmap_index *bitmap = NULL;\n-\tstruct odb_source *source;\n-\n-\t/*\n-\t * TODO: we still need to tap into implementation details of the object\n-\t * database sources. Ideally, we should extend `odb_for_each_object()`\n-\t * to handle object filters itself so that we can move the filtering\n-\t * logic into the individual sources.\n-\t */\n-\todb_prepare_alternates(the_repository->objects);\n-\tfor (source = the_repository->objects->sources; source; source = source->next) {\n-\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\tint ret = odb_source_for_each_object(&files->loose->base, NULL, batch_one_object_oi,\n-\t\t\t\t\t\t     &payload, &opts);\n-\t\tif (ret)\n-\t\t\tbreak;\n-\t}\n-\n-\tif (opt->objects_filter.choice != LOFC_DISABLED &&\n-\t    (bitmap = prepare_bitmap_git(the_repository)) &&\n-\t    !for_each_bitmapped_object(bitmap, &opt->objects_filter,\n-\t\t\t\t       batch_one_object_bitmapped, &payload)) {\n-\t\tstruct packed_git *pack;\n-\n-\t\trepo_for_each_pack(the_repository, pack) {\n-\t\t\tif (bitmap_index_contains_pack(bitmap, pack) ||\n-\t\t\t    open_pack_index(pack))\n-\t\t\t\tcontinue;\n-\t\t\tfor_each_object_in_pack(pack, batch_one_object_packed,\n-\t\t\t\t\t\t&payload, flags);\n-\t\t}\n-\t} else {\n-\t\tstruct odb_source_info source_info;\n-\t\tstruct object_info oi = {\n-\t\t\t.source_infop = &source_info,\n-\t\t};\n-\n-\t\tfor (source = the_repository->objects->sources; source; source = source->next) {\n-\t\t\tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\t\t\tint ret = odb_source_for_each_object(&files->packed->base, &oi,\n-\t\t\t\t\t\t\t     batch_one_object_oi, &payload, &opts);\n-\t\t\tif (ret)\n-\t\t\t\tbreak;\n-\t\t}\n-\t}\n \n-\tfree_bitmap_index(bitmap);\n+\todb_for_each_object_ext(the_repository->objects, &oi,\n+\t\t\t\tbatch_one_object_oi, &payload, &opts);\n }\n \n static int batch_objects(struct batch_options *opt)\n\n-- \n2.55.0.313.g8d093f411d.dirty\n\n"},{"id":"548276","messageId":"874ii0h2uf.fsf@emacs.iotcl.com","threadId":"65959","inReplyTo":"20260715-pks-odb-for-each-object-filter-v4-0-616d7adf7fb7@pks.im","subject":"Re: [PATCH v4 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2026-07-15T12:08:56Z","receivedAt":"2026-07-15T12:09:03Z","isPatch":true,"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> Hi,\n>\n> this patch series introduces object filters to `odb_for_each_object()`.\n> The intent of this is to make `git cat-file --batch-all-objects` work\n> with pluggable object databases. Right now it doesn't because it reaches\n> into internals of the \"packed\" backend to efficiently handle bitmapped\n> objects.\n>\n> The series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n> 2026-07-06) with ps/odb-drop-whence at 8a7ad23e11 (odb: document object\n> info fields, 2026-07-02) merged into it.\n>\n> Changes in v4:\n>   - Fix references to an old function name in commit messages.\n\nThanks for fixing that! This version looks fine by me.\n\n-- \nCheers,\nToon\n"},{"id":"548281","messageId":"xmqqo6g8p9mb.fsf@gitster.g","threadId":"65959","inReplyTo":"874ii0h2uf.fsf@emacs.iotcl.com","subject":"Re: [PATCH v4 0/9] odb: introduce object filters to `odb_for_each_object()`","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-07-15T15:15:24Z","receivedAt":"2026-07-15T15:15:28Z","isPatch":true,"body":"Toon Claes <toon@iotcl.com> writes:\n\n> Patrick Steinhardt <ps@pks.im> writes:\n>\n>> Hi,\n>>\n>> this patch series introduces object filters to `odb_for_each_object()`.\n>> The intent of this is to make `git cat-file --batch-all-objects` work\n>> with pluggable object databases. Right now it doesn't because it reaches\n>> into internals of the \"packed\" backend to efficiently handle bitmapped\n>> objects.\n>>\n>> The series is built on top of f85a7e6620 (Start Git 2.56 cycle,\n>> 2026-07-06) with ps/odb-drop-whence at 8a7ad23e11 (odb: document object\n>> info fields, 2026-07-02) merged into it.\n>>\n>> Changes in v4:\n>>   - Fix references to an old function name in commit messages.\n>\n> Thanks for fixing that! This version looks fine by me.\n\nGreat.  Thanks all for working well together.\n"}]}