git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 0/4] connected: search promisor objects generically

From
Patrick Steinhardt <ps@pks.im>
Date
Jun 25, 2026, 09:57 UTC
Message-ID
<20260625-pks-connected-generic-promisor-checks-v3-0-7308f3b9dc44@pks.im>
In-Reply-To
<20260622-pks-connected-generic-promisor-checks-v1-0-25eba2698202@pks.im>
Hi,

this patch series refactors "connected.c" so that we search for promisor objects in a generic way instead of reaching into internal of the object database. As a result, the connectivity checks will work properly in repos that don't use packfiles in the first place.

The series is built on top of 8d96f09e92 (Merge branch 'js/objects-larger-than-4gb-on-windows', 2026-06-19) with ps/odb-source-packed at 1bba3c035d (odb/source-packed: drop pointer to "files" parent source, 2026-06-17) merged into it.

Changes in v3:
  - Fix reversed logic for whether the promised object was found, which
    broke in v2.
  - Add a test that verifies that we indeed use the optimized check.
  - Match the hash before computing the flags so that we break out of
    the loop more eagerly.
  - Link to v2: https://patch.msgid.link/20260624-pks-connected-generic-promisor-checks-v2-0-132d73ee47b9@pks.im
Changes in v2:
  - Fix the accidentally-dropped call to `odb_reprepare()`.
  - Add a preparatory commit that splits out `check_connected_promisor()`.
    I think also splitting out `check_connected_rev_list()` would only
    have diminishing returns, so I skipped that part.
  - Link to v1: https://patch.msgid.link/20260622-pks-connected-generic-promisor-checks-v1-0-25eba2698202@pks.im
Thanks!
Patrick
---
Patrick Steinhardt (4):
      odb/source-packed: extract logic to skip certain packs
      odb/source-packed: support flags when iterating an object prefix
      connected: split out promisor-based connectivity check
      connected: search promisor objects generically
 connected.c              | 98 +++++++++++++++++++++++++++++++-----------------
 odb/source-packed.c      | 50 +++++++++++++++++-------
 t/t5616-partial-clone.sh | 24 ++++++++++++
 3 files changed, 125 insertions(+), 47 deletions(-)
Range-diff versus v2:
1:  74d1d04183 = 1:  93b7b3b4cb odb/source-packed: extract logic to skip certain packs
2:  02aa39bf1e ! 2:  3fd0885b85 odb/source-packed: support flags when iterating an object prefix
    @@ odb/source-packed.c: static int for_each_prefixed_object_in_midx(
      
      	for (; m; m = m->base_midx) {
     @@ odb/source-packed.c: static int for_each_prefixed_object_in_midx(
    - 			const struct object_id *current = NULL;
    - 			struct object_id oid;
    + 			if (!match_hash(len, opts->prefix->hash, current->hash))
    + 				break;
      
     +			if (opts->flags) {
     +				uint32_t pack_id = nth_midxed_pack_int_id(m, i);
    @@ odb/source-packed.c: static int for_each_prefixed_object_in_midx(
     +					continue;
     +			}
     +
    - 			current = nth_midxed_object_oid(&oid, m, i);
    + 			if (data->request) {
    + 				struct object_info oi = *data->request;
      
    - 			if (!match_hash(len, opts->prefix->hash, current->hash))
     @@ odb/source-packed.c: static int for_each_prefixed_object_in_midx(
      	ret = 0;
      
3:  ff9df84f65 = 3:  47a4732daf connected: split out promisor-based connectivity check
4:  a10d2e6a1e ! 4:  239abf2731 connected: search promisor objects generically
    @@ Commit message
         flag. This will yield all objects that have the exact object name and
         that are part of a promisor pack in a generic way.
     
    +    Add a test to verify that we indeed use the optimization.
    +
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## connected.c ##
    @@ connected.c
      
     +static int promised_object_cb(const struct object_id *oid UNUSED,
     +			      struct object_info *oi UNUSED,
    -+			      void *payload UNUSED)
    ++			      void *payload)
     +{
    ++	bool *found = payload;
    ++	*found = true;
     +	return 1;
     +}
     +
    @@ connected.c: static int check_connected_promisor(oid_iterate_fn fn,
      	odb_reprepare(the_repository->objects);
      	do {
     -		struct packed_git *p;
    -+		opts.prefix = *oid;
    ++		bool found = false;
      
     -		repo_for_each_pack(the_repository, p) {
     -			if (!p->pack_promisor)
    @@ connected.c: static int check_connected_promisor(oid_iterate_fn fn,
     -			if (find_pack_entry_one(*oid, p))
     -				goto promisor_pack_found;
     -		}
    -+		err = odb_for_each_object_ext(the_repository->objects,
    -+					      NULL, promised_object_cb,
    -+					      NULL, &opts);
    ++		opts.prefix = *oid;
    ++
    ++		err = odb_for_each_object_ext(the_repository->objects, NULL,
    ++					      promised_object_cb, &found, &opts);
     +		if (err < 0)
     +			return err;
      
    @@ connected.c: static int check_connected_promisor(oid_iterate_fn fn,
     -
     -promisor_pack_found:
     -		;
    -+		if (err > 0)
    ++		if (!found)
     +			return 0;
      	} while ((*oid = fn(cb_data)) != NULL);
      
      	return 1;
    +
    + ## t/t5616-partial-clone.sh ##
    +@@ t/t5616-partial-clone.sh: test_expect_success 'partial fetch inherits filter settings' '
    + 	test_line_count = 5 observed
    + '
    + 
    ++test_expect_success 'partial fetch does not spawn rev-list connectivity check' '
    ++	test_when_finished "rm -rf connectivity-remote connectivity-client" &&
    ++	git init connectivity-remote &&
    ++	test_commit -C connectivity-remote one &&
    ++	git -C connectivity-remote config uploadpack.allowfilter 1 &&
    ++	git -C connectivity-remote config uploadpack.allowanysha1inwant 1 &&
    ++
    ++	git clone --no-checkout --filter=blob:none \
    ++		"file://$(pwd)/connectivity-remote" connectivity-client &&
    ++
    ++	# When doing a partial fetch where all tips are part of a promisor pack
    ++	# we want to skip the connectivity check, as these objects are allowed
    ++	# to not be fully connected.
    ++	test_commit -C connectivity-remote two &&
    ++	GIT_TRACE2_EVENT="$(pwd)/partial.trace" git -C connectivity-client fetch origin &&
    ++	test_subcommand_flex ! git rev-list --objects --stdin <partial.trace &&
    ++
    ++	# Otherwise, when doing a fetch where any of the tips is not part of a
    ++	# promisor pack, then we must run the connectivity check.
    ++	test_commit -C connectivity-remote three &&
    ++	GIT_TRACE2_EVENT="$(pwd)/full.trace" git -C connectivity-client fetch --no-filter origin &&
    ++	test_subcommand_flex git rev-list --objects --stdin <full.trace
    ++'
    ++
    + # force dynamic object fetch using diff.
    + # we should only get 1 new blob (for the file in origin/main).
    + test_expect_success 'verify diff causes dynamic object fetch' '

--- base-commit: 4a8e7a446f41435e157131162dfe901eca9250fe change-id: 20260612-pks-connected-generic-promisor-checks-2933bff3028d

Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 19 of 25 in “connected: search promisor objects generically”
  1. 0/3 connected: search promisor objects genericallyPatrick Steinhardt, Jun 22, 2026
  2. 1/3 odb/source-packed: extract logic to skip certain packsPatrick Steinhardt, Jun 22, 2026
  3. Junio C HamanoJun 22, 2026
  4. 2/3 odb/source-packed: support flags when iterating an object prefixPatrick Steinhardt, Jun 22, 2026
  5. 3/3 connected: search promisor objects genericallyPatrick Steinhardt, Jun 22, 2026
  6. Junio C HamanoJun 22, 2026
  7. Patrick SteinhardtJun 24, 2026
  8. Christian CouderJun 23, 2026
  9. Patrick SteinhardtJun 24, 2026
  10. 0/4 connected: search promisor objects genericallyPatrick Steinhardt, Jun 24, 2026
  11. 1/4 odb/source-packed: extract logic to skip certain packsPatrick Steinhardt, Jun 24, 2026
  12. 2/4 odb/source-packed: support flags when iterating an object prefixPatrick Steinhardt, Jun 24, 2026
  13. Christian CouderJun 24, 2026
  14. Patrick SteinhardtJun 25, 2026
  15. 3/4 connected: split out promisor-based connectivity checkPatrick Steinhardt, Jun 24, 2026
  16. 4/4 connected: search promisor objects genericallyPatrick Steinhardt, Jun 24, 2026
  17. Junio C HamanoJun 24, 2026
  18. Patrick SteinhardtJun 25, 2026
  19. 0/4 connected: search promisor objects genericallyPatrick Steinhardt, Jun 25, 2026
  20. 1/4 odb/source-packed: extract logic to skip certain packsPatrick Steinhardt, Jun 25, 2026
  21. 2/4 odb/source-packed: support flags when iterating an object prefixPatrick Steinhardt, Jun 25, 2026
  22. 3/4 connected: split out promisor-based connectivity checkPatrick Steinhardt, Jun 25, 2026
  23. 4/4 connected: search promisor objects genericallyPatrick Steinhardt, Jun 25, 2026
  24. Junio C HamanoJun 25, 2026
  25. Christian CouderJun 26, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.