git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 10/13] http-walker: don't fetch objects via promisor remotes

From
Patrick Steinhardt <ps@pks.im>
Date
Apr 23, 2025, 07:48 UTC
Message-ID
<20250423-pks-object-store-cleanups-v1-10-81f8411a5d08@pks.im>
In-Reply-To
<20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im>

The HTTP walker is responsible for fetching objects via the dumb HTTP protocol. To avoid re-fetching objects that we already have locally we first check whether the object already exists in the local repository before issuing the requests.

This existence check is done by calling `repo_has_object_file()`, which will fetch the object via a promisor remote in case it is in a promisor pack. This fetch does not make any sense for us though: we're already in the process of fetching the object anyway, so fetching it via a separate connection is wasteful, but should otherwise be harmless.

Fix the issue by converting to `has_object()`, which knows to not fetch objects via promisor remotes by default.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 http-walker.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/http-walker.c b/http-walker.c
index 95458e2f638..5ad2eae9a11 100644
--- a/http-walker.c
+++ b/http-walker.c
@@ -138,7 +138,7 @@ static int fill_active_slot(void *data UNUSED)
 	list_for_each_safe(pos, tmp, head) {
 		obj_req = list_entry(pos, struct object_request, node);
 		if (obj_req->state == WAITING) {
-			if (repo_has_object_file(the_repository, &obj_req->oid))
+			if (has_object(the_repository, &obj_req->oid, HAS_OBJECT_RECHECK_PACKED))
 				obj_req->state = COMPLETE;
 			else {
 				start_object_request(obj_req);
@@ -496,7 +496,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)
 	if (!obj_req)
 		return error("Couldn't find request for %s in the queue", hex);
 
-	if (repo_has_object_file(the_repository, &obj_req->oid)) {
+	if (has_object(the_repository, &obj_req->oid, HAS_OBJECT_RECHECK_PACKED)) {
 		if (obj_req->req)
 			abort_http_object_request(&obj_req->req);
 		abort_object_request(obj_req);
-- 
2.49.0.901.g37484f566f.dirty
Previous: Karthik NayakNext: Patrick Steinhardt
Message 17 of 55 in “object-store: a handful of cleanups”
  1. 00/13 object-store: a handful of cleanupsPatrick Steinhardt, Apr 23, 2025
  2. 01/13 object-store: move `struct packed_git` into "packfile.h"Patrick Steinhardt, Apr 23, 2025
  3. 02/13 object-store: drop `loose_object_path()`Patrick Steinhardt, Apr 23, 2025
  4. 03/13 object-store: move and rename `odb_pack_keep()`Patrick Steinhardt, Apr 23, 2025
  5. Karthik NayakApr 23, 2025
  6. 05/13 object-store: allow fetching objects via `has_object()`Patrick Steinhardt, Apr 23, 2025
  7. Karthik NayakApr 23, 2025
  8. 04/13 object-store: move function declarations to their respective subsystemsPatrick Steinhardt, Apr 23, 2025
  9. 06/13 treewide: trivial conversions of `repo_has_object_file()`Patrick Steinhardt, Apr 23, 2025
  10. 07/13 builtin/index-pack: don't fetch promised objects for collision checkPatrick Steinhardt, Apr 23, 2025
  11. Karthik NayakApr 23, 2025
  12. Patrick SteinhardtApr 25, 2025
  13. Karthik NayakApr 28, 2025
  14. 08/13 builtin/show-ref: don't fetch objects when printing refsPatrick Steinhardt, Apr 23, 2025
  15. 09/13 refs: don't fetch promisor objects in `ref_resolves_to_object()`Patrick Steinhardt, Apr 23, 2025
  16. Karthik NayakApr 23, 2025
  17. 10/13 http-walker: don't fetch objects via promisor remotesPatrick Steinhardt, Apr 23, 2025
  18. 11/13 list-objects: clarify how promised blobs are excludedPatrick Steinhardt, Apr 23, 2025
  19. 12/13 bulk-checkin: don't fetch promised objects on writePatrick Steinhardt, Apr 23, 2025
  20. 13/13 object-store: drop `repo_has_object_file()`Patrick Steinhardt, Apr 23, 2025
  21. Karthik NayakApr 23, 2025
  22. Patrick SteinhardtApr 25, 2025
  23. 00/13 object-store: a handful of cleanupsPatrick Steinhardt, Apr 25, 2025
  24. 01/13 object-store: move `struct packed_git` into "packfile.h"Patrick Steinhardt, Apr 25, 2025
  25. 02/13 object-store: drop `loose_object_path()`Patrick Steinhardt, Apr 25, 2025
  26. 03/13 object-store: move and rename `odb_pack_keep()`Patrick Steinhardt, Apr 25, 2025
  27. 04/13 object-store: move function declarations to their respective subsystemsPatrick Steinhardt, Apr 25, 2025
  28. 05/13 object-store: allow fetching objects via `has_object()`Patrick Steinhardt, Apr 25, 2025
  29. 06/13 treewide: trivial conversions of `repo_has_object_file()`Patrick Steinhardt, Apr 25, 2025
  30. Junio C HamanoApr 28, 2025
  31. 07/13 builtin/index-pack: don't fetch promised objects for collision checkPatrick Steinhardt, Apr 25, 2025
  32. Junio C HamanoApr 28, 2025
  33. Patrick SteinhardtApr 29, 2025
  34. 08/13 builtin/show-ref: don't fetch objects when printing refsPatrick Steinhardt, Apr 25, 2025
  35. Junio C HamanoApr 28, 2025
  36. 09/13 refs: don't fetch promisor objects in `ref_resolves_to_object()`Patrick Steinhardt, Apr 25, 2025
  37. Junio C HamanoApr 28, 2025
  38. 10/13 http-walker: don't fetch objects via promisor remotesPatrick Steinhardt, Apr 25, 2025
  39. Junio C HamanoApr 28, 2025
  40. 11/13 list-objects: clarify how promised blobs are excludedPatrick Steinhardt, Apr 25, 2025
  41. 12/13 bulk-checkin: don't fetch promised objects on writePatrick Steinhardt, Apr 25, 2025
  42. Junio C HamanoApr 28, 2025
  43. Patrick SteinhardtApr 29, 2025
  44. Junio C HamanoApr 29, 2025
  45. 13/13 object-store: drop `repo_has_object_file()`Patrick Steinhardt, Apr 25, 2025
  46. Karthik NayakApr 28, 2025
  47. 0/7 object-store: a handful of cleanupsPatrick Steinhardt, Apr 29, 2025
  48. 1/7 object-store: move `struct packed_git` into "packfile.h"Patrick Steinhardt, Apr 29, 2025
  49. 2/7 object-store: drop `loose_object_path()`Patrick Steinhardt, Apr 29, 2025
  50. 3/7 object-store: move and rename `odb_pack_keep()`Patrick Steinhardt, Apr 29, 2025
  51. 4/7 object-store: move function declarations to their respective subsystemsPatrick Steinhardt, Apr 29, 2025
  52. 5/7 object-store: allow fetching objects via `has_object()`Patrick Steinhardt, Apr 29, 2025
  53. 6/7 treewide: convert users of `repo_has_object_file()` to `has_object()`Patrick Steinhardt, Apr 29, 2025
  54. 7/7 object-store: drop `repo_has_object_file()`Patrick Steinhardt, Apr 29, 2025
  55. Junio C HamanoApr 29, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.