From: Patrick Steinhardt Date: Fri, 20 Mar 2026 07:07:26 GMT Subject: [PATCH v2 00/14] odb: generic object name handling Message-ID: <20260320-b4-pks-odb-source-abbrev-v2-0-fe65dcd8c735@pks.im> In-Reply-To: <20260319-b4-pks-odb-source-abbrev-v1-0-5ddebad292b0@pks.im> Hi, this patch series refactors handling of object names to become pluggable and thus generic. This includes: - Disambiguation of object names with a common prefix. This is required to list candidate objects in case the user has passed a non-unique prefix. - Abbreviating an object ID to the shortest prefix required while staying unique. The logic to compute these operations is specific to the backend, but not generic. This patch series fixes that by moving the functionality into the respective backends. This patch series may feel somewhat unexiting, but it's not. Especially abbreviating object IDs is done in lots of places, so this functionality is overall quite critical. So starting with this series, it is now possible to do all kinds of local work with an alternative backend: git-commit(1), git-log(1), git-rev-parse(1), git-merge(1) and many other commands now work as expected. My MongoDB proof of concept [1] only requires two commits (the object format extension) on top. And no, I don't endorse MongoDB or propose it as a future potential backend. It simply had a good C API that was easy to use. Of course, other functionality, especially everything that involves packfiles, doesn't yet work. This patch series is built on top of ca1db8a0f7 (The 17th batch, 2026-03-16) with ps/object-counting at 6801ffd37d (odb: introduce generic object counting, 2026-03-12) merged into it. Changes in v2: - Document `cb_iter` callback. - Fix left-over conversion of `odb_source_loose_for_each_object()`. - commit message typo fixes. - Link to v1: https://lore.kernel.org/r/20260319-b4-pks-odb-source-abbrev-v1-0-5ddebad292b0@pks.im Thanks! Patrick [1]: https://gitlab.com/gitlab-org/git/-/merge_requests/454 --- Patrick Steinhardt (14): oidtree: modernize the code a bit oidtree: extend iteration to allow for arbitrary return codes odb: introduce `struct odb_for_each_object_options` object-name: move logic to iterate through loose prefixed objects object-name: move logic to iterate through packed prefixed objects object-name: extract function to parse object ID prefixes object-name: backend-generic `repo_collect_ambiguous()` object-name: backend-generic `get_short_oid()` object-name: merge `update_candidates()` and `match_prefix()` object-name: abbreviate loose object names without `disambiguate_state` object-name: simplify computing common prefixes object-name: move logic to compute loose abbreviation length object-file: move logic to compute packed abbreviation length odb: introduce generic `odb_find_abbrev_len()` builtin/cat-file.c | 7 +- builtin/pack-objects.c | 12 +- cbtree.c | 21 ++- cbtree.h | 17 +- commit-graph.c | 5 +- hash.c | 18 ++ hash.h | 3 + object-file.c | 76 ++++++++- object-file.h | 21 ++- object-name.c | 437 ++++++++--------------------------------------- odb.c | 99 ++++++++++- odb.h | 39 +++++ odb/source-files.c | 33 +++- odb/source.h | 30 +++- oidtree.c | 65 +++---- oidtree.h | 48 +++++- packfile.c | 297 +++++++++++++++++++++++++++++++- packfile.h | 7 +- t/unit-tests/u-oidtree.c | 18 +- 19 files changed, 782 insertions(+), 471 deletions(-) Range-diff versus v1: 1: 5ce8cced1e = 1: 755acf126c oidtree: modernize the code a bit 2: 26c4377ff1 ! 2: 6ab9e5d41c oidtree: extend iteration to allow for arbitrary return codes @@ cbtree.h: static inline void cb_init(struct cb_tree *t) struct cb_node *cb_insert(struct cb_tree *, struct cb_node *, size_t klen); -typedef enum cb_next (*cb_iter)(struct cb_node *, void *arg); ++/* ++ * Callback invoked by `cb_each()` for each node in the critbit tree. A return ++ * value of 0 will cause the iteration to continue, a non-zero return code will ++ * cause iteration to abort. The error code will be relayed back from ++ * `cb_each()` in that case. ++ */ +typedef int (*cb_iter)(struct cb_node *, void *arg); -void cb_each(struct cb_tree *, const uint8_t *kpfx, size_t klen, 3: da7b74b572 ! 3: 9caf0288e4 odb: introduce `struct odb_for_each_object_options` @@ Commit message a subsequent commit we'll want to change object iteration to also support iterating over only those objects that have a specific prefix. While we could of course add the prefix to the function signature, or - alternative introduce a new function, both of these options don't really - seem to be that sensible. + alternatively introduce a new function, both of these options don't + really seem to be that sensible. Instead, introduce a new `struct odb_for_each_object_options` that can be passed to a new `odb_for_each_object_ext()` function. Splice through @@ object-file.c: int odb_source_loose_for_each_object(struct odb_source *source, return 0; return for_each_loose_file_in_source(source, for_each_object_wrapper_cb, +@@ object-file.c: int odb_source_loose_count_objects(struct odb_source *source, + *out = count * 256; + ret = 0; + } else { ++ struct odb_for_each_object_options opts = { 0 }; + *out = 0; + ret = odb_source_loose_for_each_object(source, NULL, count_loose_object, +- out, 0); ++ out, &opts); + } + + out: ## object-file.h ## @@ object-file.h: int odb_source_loose_for_each_object(struct odb_source *source, 4: e4dedd4686 = 4: 232bcf662e object-name: move logic to iterate through loose prefixed objects 5: 7f9bfca9fd = 5: be9b546c67 object-name: move logic to iterate through packed prefixed objects 6: 3eb88fa774 = 6: 60672e6cd1 object-name: extract function to parse object ID prefixes 7: 8fa16eb02a ! 7: 38032fee38 object-name: backend-generic `repo_collect_ambiguous()` @@ Commit message to do is to enumerate objects that have such a prefix and then append those objects to a `struct oid_array`. This can be trivially achieved in a generic way now that `odb_for_each_object()` has learned to yield - only objects that much such a prefix. + only objects that match such a prefix. Refactor the code to use the backend-generic infrastructure instead. 8: 0b0ce71bdd = 8: 98af219ca1 object-name: backend-generic `get_short_oid()` 9: 74733d34de = 9: 12866c582b object-name: merge `update_candidates()` and `match_prefix()` 10: 2ca53a01d4 = 10: 6b4eda123a object-name: abbreviate loose object names without `disambiguate_state` 11: a391911854 = 11: 7aa921504b object-name: simplify computing common prefixes 12: 9ad66df1ca = 12: d1bf2706f8 object-name: move logic to compute loose abbreviation length 13: cf8a33ab67 = 13: e59e7dffb4 object-file: move logic to compute packed abbreviation length 14: acd07686db = 14: 726be6de40 odb: introduce generic `odb_find_abbrev_len()` --- base-commit: b052aca69d64d2d8e28e7ce97dcb1beb3d94515a change-id: 20260313-b4-pks-odb-source-abbrev-a84c51222bca