From: Patrick Steinhardt Date: Thu, 25 Jun 2026 09:57:38 GMT Subject: [PATCH v3 0/4] connected: search promisor objects generically Message-ID: <20260625-pks-connected-generic-promisor-checks-v3-0-7308f3b9dc44@pks.im> In-Reply-To: <20260622-pks-connected-generic-promisor-checks-v1-0-25eba2698202@pks.im> Hi, this patch series refactors "connected.c" so that we search for promisor objects in a generic way instead of reaching into internal of the object database. As a result, the connectivity checks will work properly in repos that don't use packfiles in the first place. The series is built on top of 8d96f09e92 (Merge branch 'js/objects-larger-than-4gb-on-windows', 2026-06-19) with ps/odb-source-packed at 1bba3c035d (odb/source-packed: drop pointer to "files" parent source, 2026-06-17) merged into it. Changes in v3: - Fix reversed logic for whether the promised object was found, which broke in v2. - Add a test that verifies that we indeed use the optimized check. - Match the hash before computing the flags so that we break out of the loop more eagerly. - Link to v2: https://patch.msgid.link/20260624-pks-connected-generic-promisor-checks-v2-0-132d73ee47b9@pks.im Changes in v2: - Fix the accidentally-dropped call to `odb_reprepare()`. - Add a preparatory commit that splits out `check_connected_promisor()`. I think also splitting out `check_connected_rev_list()` would only have diminishing returns, so I skipped that part. - Link to v1: https://patch.msgid.link/20260622-pks-connected-generic-promisor-checks-v1-0-25eba2698202@pks.im Thanks! Patrick --- Patrick Steinhardt (4): odb/source-packed: extract logic to skip certain packs odb/source-packed: support flags when iterating an object prefix connected: split out promisor-based connectivity check connected: search promisor objects generically connected.c | 98 +++++++++++++++++++++++++++++++----------------- odb/source-packed.c | 50 +++++++++++++++++------- t/t5616-partial-clone.sh | 24 ++++++++++++ 3 files changed, 125 insertions(+), 47 deletions(-) Range-diff versus v2: 1: 74d1d04183 = 1: 93b7b3b4cb odb/source-packed: extract logic to skip certain packs 2: 02aa39bf1e ! 2: 3fd0885b85 odb/source-packed: support flags when iterating an object prefix @@ odb/source-packed.c: static int for_each_prefixed_object_in_midx( for (; m; m = m->base_midx) { @@ odb/source-packed.c: static int for_each_prefixed_object_in_midx( - const struct object_id *current = NULL; - struct object_id oid; + if (!match_hash(len, opts->prefix->hash, current->hash)) + break; + if (opts->flags) { + uint32_t pack_id = nth_midxed_pack_int_id(m, i); @@ odb/source-packed.c: static int for_each_prefixed_object_in_midx( + continue; + } + - current = nth_midxed_object_oid(&oid, m, i); + if (data->request) { + struct object_info oi = *data->request; - if (!match_hash(len, opts->prefix->hash, current->hash)) @@ odb/source-packed.c: static int for_each_prefixed_object_in_midx( ret = 0; 3: ff9df84f65 = 3: 47a4732daf connected: split out promisor-based connectivity check 4: a10d2e6a1e ! 4: 239abf2731 connected: search promisor objects generically @@ Commit message flag. This will yield all objects that have the exact object name and that are part of a promisor pack in a generic way. + Add a test to verify that we indeed use the optimization. + Signed-off-by: Patrick Steinhardt ## connected.c ## @@ connected.c +static int promised_object_cb(const struct object_id *oid UNUSED, + struct object_info *oi UNUSED, -+ void *payload UNUSED) ++ void *payload) +{ ++ bool *found = payload; ++ *found = true; + return 1; +} + @@ connected.c: static int check_connected_promisor(oid_iterate_fn fn, odb_reprepare(the_repository->objects); do { - struct packed_git *p; -+ opts.prefix = *oid; ++ bool found = false; - repo_for_each_pack(the_repository, p) { - if (!p->pack_promisor) @@ connected.c: static int check_connected_promisor(oid_iterate_fn fn, - if (find_pack_entry_one(*oid, p)) - goto promisor_pack_found; - } -+ err = odb_for_each_object_ext(the_repository->objects, -+ NULL, promised_object_cb, -+ NULL, &opts); ++ opts.prefix = *oid; ++ ++ err = odb_for_each_object_ext(the_repository->objects, NULL, ++ promised_object_cb, &found, &opts); + if (err < 0) + return err; @@ connected.c: static int check_connected_promisor(oid_iterate_fn fn, - -promisor_pack_found: - ; -+ if (err > 0) ++ if (!found) + return 0; } while ((*oid = fn(cb_data)) != NULL); return 1; + + ## t/t5616-partial-clone.sh ## +@@ t/t5616-partial-clone.sh: test_expect_success 'partial fetch inherits filter settings' ' + test_line_count = 5 observed + ' + ++test_expect_success 'partial fetch does not spawn rev-list connectivity check' ' ++ test_when_finished "rm -rf connectivity-remote connectivity-client" && ++ git init connectivity-remote && ++ test_commit -C connectivity-remote one && ++ git -C connectivity-remote config uploadpack.allowfilter 1 && ++ git -C connectivity-remote config uploadpack.allowanysha1inwant 1 && ++ ++ git clone --no-checkout --filter=blob:none \ ++ "file://$(pwd)/connectivity-remote" connectivity-client && ++ ++ # When doing a partial fetch where all tips are part of a promisor pack ++ # we want to skip the connectivity check, as these objects are allowed ++ # to not be fully connected. ++ test_commit -C connectivity-remote two && ++ GIT_TRACE2_EVENT="$(pwd)/partial.trace" git -C connectivity-client fetch origin && ++ test_subcommand_flex ! git rev-list --objects --stdin