{"thread":{"id":"63335","subject":"[PATCH 00/13] object-store: a handful of cleanups","startedAt":"2025-04-23T07:48:26Z","lastAt":"2025-04-29T20:07:38Z","messageCount":55,"participants":["Patrick Steinhardt","Karthik Nayak","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":13},"messages":[{"id":"516557","messageId":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":null,"subject":"[PATCH 00/13] object-store: a handful of cleanups","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:10Z","receivedAt":"2025-04-23T07:48:26Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series contains a handful of cleanups to the object store\nsubsystem:\n\n - A couple of definitions are moved out of \"object-store.h\" as they\n belong to other subsystems.\n\n - Some functions are dropped and/or renamed.\n\n - The biggest part is the removal of `repo_has_object_file()`. This\n function and its `_with_flags()` variant are marked as deprecated,\n with the replacement being `has_object()`. The benefit of that\n function is that it doesn't reload packfiles and doesn't fetch\n promisor objects by default so that it becomes more explicit when\n one really wants to do so.\n\nThese cleanups are in preparation for getting rid of `the_repository` in\n\"object-store.c\".\n\nThe patch series is built on top of 4bbb303af69 (The seventh batch,\n2025-04-17) with ps/object-file-cleanup at 68cd492a3e6 (object-store:\nmerge \"object-store-ll.h\" and \"object-store.h\", 2025-04-15) merged into\nit.\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (13):\n object-store: move `struct packed_git` into \"packfile.h\"\n object-store: drop `loose_object_path()`\n object-store: move and rename `odb_pack_keep()`\n object-store: move function declarations to their respective subsystems\n object-store: allow fetching objects via `has_object()`\n treewide: trivial conversions of `repo_has_object_file()`\n builtin/index-pack: don't fetch promised objects for collision check\n builtin/show-ref: don't fetch objects when printing refs\n refs: don't fetch promisor objects in `ref_resolves_to_object()`\n http-walker: don't fetch objects via promisor remotes\n list-objects: clarify how promised blobs are excluded\n bulk-checkin: don't fetch promised objects on write\n object-store: drop `repo_has_object_file()`\n\n builtin/cat-file.c | 3 +-\n builtin/clone.c | 4 +-\n builtin/count-objects.c | 2 +-\n builtin/fast-import.c | 3 +-\n builtin/fetch.c | 15 ++--\n builtin/gc.c | 2 +-\n builtin/index-pack.c | 6 +-\n builtin/receive-pack.c | 4 +-\n builtin/remote.c | 3 +-\n builtin/show-ref.c | 2 +-\n builtin/unpack-objects.c | 3 +-\n bulk-checkin.c | 2 +-\n cache-tree.c | 13 +++-\n fetch-pack.c | 7 +-\n http-push.c | 11 ++-\n http-walker.c | 7 +-\n http.c | 4 +-\n list-objects.c | 3 +-\n notes.c | 3 +-\n object-file.c | 4 +-\n object-file.h | 77 +++++++++++++++++++\n object-name.c | 2 +-\n object-store.c | 44 ++---------\n object-store.h | 191 +++--------------------------------------------\n pack-objects.h | 1 +\n packfile.h | 78 ++++++++++++++++++-\n path.c | 14 ++++\n path.h | 7 ++\n prune-packed.c | 2 +-\n reachable.c | 2 +-\n reflog.c | 3 +-\n refs.c | 2 +-\n remote.c | 2 +-\n send-pack.c | 5 +-\n shallow.c | 9 ++-\n upload-pack.c | 3 +-\n walker.c | 3 +-\n 37 files changed, 265 insertions(+), 281 deletions(-)\n\n\n---\nbase-commit: ca819c0751cedd1713334882e4c83687f8478a54\nchange-id: 20250422-pks-object-store-cleanups-5a6077014155\n\n"},{"id":"516558","messageId":"20250423-pks-object-store-cleanups-v1-1-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 01/13] object-store: move `struct packed_git` into \"packfile.h\"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:11Z","receivedAt":"2025-04-23T07:48:26Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The \"object-store.h\" header contains the definition of `struct\npacked_git`. As this structure hosts all kind of information about a\nspecific packfile it is arguably a bit out of place in a generic place\nlike \"object-store.h\".\n\nMove the structure as well as `pack_map_entry_cmp()` into \"packfile.h\".\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.h | 59 +---------------------------------------------------------\n pack-objects.h | 1 +\n packfile.h | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-\n 3 files changed, 60 insertions(+), 59 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 46961dc9542..e04469a85fb 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -92,65 +92,8 @@ struct oidtree *odb_loose_cache(struct object_directory *odb,\n /* Empty the loose object cache for the specified object directory. */\n void odb_clear_loose_cache(struct object_directory *odb);\n \n-struct packed_git {\n-\tstruct hashmap_entry packmap_ent;\n-\tstruct packed_git *next;\n-\tstruct list_head mru;\n-\tstruct pack_window *windows;\n-\toff_t pack_size;\n-\tconst void *index_data;\n-\tsize_t index_size;\n-\tuint32_t num_objects;\n-\tsize_t crc_offset;\n-\tstruct oidset bad_objects;\n-\tint index_version;\n-\ttime_t mtime;\n-\tint pack_fd;\n-\tint index; /* for builtin/pack-objects.c */\n-\tunsigned pack_local:1,\n-\t\t pack_keep:1,\n-\t\t pack_keep_in_core:1,\n-\t\t freshened:1,\n-\t\t do_not_close:1,\n-\t\t pack_promisor:1,\n-\t\t multi_pack_index:1,\n-\t\t is_cruft:1;\n-\tunsigned char hash[GIT_MAX_RAWSZ];\n-\tstruct revindex_entry *revindex;\n-\tconst uint32_t *revindex_data;\n-\tconst uint32_t *revindex_map;\n-\tsize_t revindex_size;\n-\t/*\n-\t * mtimes_map points at the beginning of the memory mapped region of\n-\t * this pack's corresponding .mtimes file, and mtimes_size is the size\n-\t * of that .mtimes file\n-\t */\n-\tconst uint32_t *mtimes_map;\n-\tsize_t mtimes_size;\n-\n-\t/* repo denotes the repository this packfile belongs to */\n-\tstruct repository *repo;\n-\n-\t/* something like \".git/objects/pack/xxxxx.pack\" */\n-\tchar pack_name[FLEX_ARRAY]; /* more */\n-};\n-\n+struct packed_git;\n struct multi_pack_index;\n-\n-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,\n-\t\t\t\t const struct hashmap_entry *entry,\n-\t\t\t\t const struct hashmap_entry *entry2,\n-\t\t\t\t const void *keydata)\n-{\n-\tconst char *key = keydata;\n-\tconst struct packed_git *pg1, *pg2;\n-\n-\tpg1 = container_of(entry, const struct packed_git, packmap_ent);\n-\tpg2 = container_of(entry2, const struct packed_git, packmap_ent);\n-\n-\treturn strcmp(pg1->pack_name, key ? key : pg2->pack_name);\n-}\n-\n struct cached_object_entry;\n \n struct raw_object_store {\ndiff --git a/pack-objects.h b/pack-objects.h\nindex d1c4ae7f9b6..475a2d67ce3 100644\n--- a/pack-objects.h\n+++ b/pack-objects.h\n@@ -4,6 +4,7 @@\n #include \"object-store.h\"\n #include \"thread-utils.h\"\n #include \"pack.h\"\n+#include \"packfile.h\"\n \n struct repository;\n \ndiff --git a/packfile.h b/packfile.h\nindex 25097213d06..05499382397 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -1,13 +1,70 @@\n #ifndef PACKFILE_H\n #define PACKFILE_H\n \n+#include \"list.h\"\n #include \"object.h\"\n #include \"oidset.h\"\n \n /* in object-store.h */\n-struct packed_git;\n struct object_info;\n \n+struct packed_git {\n+\tstruct hashmap_entry packmap_ent;\n+\tstruct packed_git *next;\n+\tstruct list_head mru;\n+\tstruct pack_window *windows;\n+\toff_t pack_size;\n+\tconst void *index_data;\n+\tsize_t index_size;\n+\tuint32_t num_objects;\n+\tsize_t crc_offset;\n+\tstruct oidset bad_objects;\n+\tint index_version;\n+\ttime_t mtime;\n+\tint pack_fd;\n+\tint index; /* for builtin/pack-objects.c */\n+\tunsigned pack_local:1,\n+\t\t pack_keep:1,\n+\t\t pack_keep_in_core:1,\n+\t\t freshened:1,\n+\t\t do_not_close:1,\n+\t\t pack_promisor:1,\n+\t\t multi_pack_index:1,\n+\t\t is_cruft:1;\n+\tunsigned char hash[GIT_MAX_RAWSZ];\n+\tstruct revindex_entry *revindex;\n+\tconst uint32_t *revindex_data;\n+\tconst uint32_t *revindex_map;\n+\tsize_t revindex_size;\n+\t/*\n+\t * mtimes_map points at the beginning of the memory mapped region of\n+\t * this pack's corresponding .mtimes file, and mtimes_size is the size\n+\t * of that .mtimes file\n+\t */\n+\tconst uint32_t *mtimes_map;\n+\tsize_t mtimes_size;\n+\n+\t/* repo denotes the repository this packfile belongs to */\n+\tstruct repository *repo;\n+\n+\t/* something like \".git/objects/pack/xxxxx.pack\" */\n+\tchar pack_name[FLEX_ARRAY]; /* more */\n+};\n+\n+static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,\n+\t\t\t\t const struct hashmap_entry *entry,\n+\t\t\t\t const struct hashmap_entry *entry2,\n+\t\t\t\t const void *keydata)\n+{\n+\tconst char *key = keydata;\n+\tconst struct packed_git *pg1, *pg2;\n+\n+\tpg1 = container_of(entry, const struct packed_git, packmap_ent);\n+\tpg2 = container_of(entry2, const struct packed_git, packmap_ent);\n+\n+\treturn strcmp(pg1->pack_name, key ? key : pg2->pack_name);\n+}\n+\n struct pack_window {\n \tstruct pack_window *next;\n \tunsigned char *base;\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516559","messageId":"20250423-pks-object-store-cleanups-v1-2-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 02/13] object-store: drop `loose_object_path()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:12Z","receivedAt":"2025-04-23T07:48:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The function `loose_object_path()` is a trivial wrapper around\n`odb_loose_path()`, with the only exception that it always uses the\nprimary object database of the given repository. This doesn't really add\na ton of value though, so let's drop the function and inline it at every\ncallsite.\n\nSigned-off-by: Patrick Steinhardt \n---\n http-walker.c | 3 ++-\n http.c | 4 ++--\n object-file.c | 4 ++--\n object-file.h | 4 ++++\n object-store.c | 6 ------\n object-store.h | 7 -------\n 6 files changed, 10 insertions(+), 18 deletions(-)\n\ndiff --git a/http-walker.c b/http-walker.c\nindex 882cae19c24..95458e2f638 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -9,6 +9,7 @@\n #include \"list.h\"\n #include \"transport.h\"\n #include \"packfile.h\"\n+#include \"object-file.h\"\n #include \"object-store.h\"\n \n struct alt_base {\n@@ -540,7 +541,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)\n \t\tret = error(\"File %s has bad hash\", hex);\n \t} else if (req->rename < 0) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tloose_object_path(the_repository, &buf, &req->oid);\n+\t\todb_loose_path(the_repository->objects->odb, &buf, &req->oid);\n \t\tret = error(\"unable to write sha1 filename %s\", buf.buf);\n \t\tstrbuf_release(&buf);\n \t}\ndiff --git a/http.c b/http.c\nindex 0c411380425..3c029cf8947 100644\n--- a/http.c\n+++ b/http.c\n@@ -2662,7 +2662,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \toidcpy(&freq->oid, oid);\n \tfreq->localfile = -1;\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \tstrbuf_addf(&freq->tmpfile, \"%s.temp\", filename.buf);\n \n \tstrbuf_addf(&prevfile, \"%s.prev\", filename.buf);\n@@ -2814,7 +2814,7 @@ int finish_http_object_request(struct http_object_request *freq)\n \t\tunlink_or_warn(freq->tmpfile.buf);\n \t\treturn -1;\n \t}\n-\tloose_object_path(the_repository, &filename, &freq->oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, &freq->oid);\n \tfreq->rename = finalize_object_file(freq->tmpfile.buf, filename.buf);\n \tstrbuf_release(&filename);\n \ndiff --git a/object-file.c b/object-file.c\nindex 9cc3a24a40d..dc56a4766df 100644\n--- a/object-file.c\n+++ b/object-file.c\n@@ -932,7 +932,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr,\n \tif (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT))\n \t\tprepare_loose_object_bulk_checkin();\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \n \tfd = start_loose_object_common(&tmp_file, filename.buf, flags,\n \t\t\t\t &stream, compressed, sizeof(compressed),\n@@ -1079,7 +1079,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len,\n \t\tgoto cleanup;\n \t}\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \n \t/* We finally know the object path, and create the missing dir. */\n \tdirlen = directory_size(filename.buf);\ndiff --git a/object-file.h b/object-file.h\nindex c002fbe2345..0a7b6b9f9d9 100644\n--- a/object-file.h\n+++ b/object-file.h\n@@ -25,6 +25,10 @@ int index_path(struct index_state *istate, struct object_id *oid, const char *pa\n \n struct object_directory;\n \n+/*\n+ * Put in `buf` the name of the file in the local object database that\n+ * would be used to store a loose object with the specified oid.\n+ */\n const char *odb_loose_path(struct object_directory *odb,\n \t\t\t struct strbuf *buf,\n \t\t\t const struct object_id *oid);\ndiff --git a/object-store.c b/object-store.c\nindex 6ab50d25d3e..e5cfb8c0079 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -96,12 +96,6 @@ int odb_pack_keep(const char *name)\n \treturn open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n }\n \n-const char *loose_object_path(struct repository *r, struct strbuf *buf,\n-\t\t\t const struct object_id *oid)\n-{\n-\treturn odb_loose_path(r->objects->odb, buf, oid);\n-}\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\ndiff --git a/object-store.h b/object-store.h\nindex e04469a85fb..5668de62d01 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -196,13 +196,6 @@ int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n */\n int odb_pack_keep(const char *name);\n \n-/*\n- * Put in `buf` the name of the file in the local object database that\n- * would be used to store a loose object with the specified oid.\n- */\n-const char *loose_object_path(struct repository *r, struct strbuf *buf,\n-\t\t\t const struct object_id *oid);\n-\n void *map_loose_object(struct repository *r, const struct object_id *oid,\n \t\t unsigned long *size);\n \n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516560","messageId":"20250423-pks-object-store-cleanups-v1-3-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 03/13] object-store: move and rename `odb_pack_keep()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:13Z","receivedAt":"2025-04-23T07:48:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The function `odb_pack_keep()` creates a file at the passed-in path. If\nthis fails, then the function re-tries by first creating any potentially\nmissing leading directoriesk and then trying to create the file once\nagain. As such, this function doesn't host any kind of logic that is\nspecific to the object store, but is rather a generic helper function.\n\nRename the function to `safe_create_file_with_leading_directories()` and\nmove it into \"path.c\". While at it, refactor it so that it loses its\ndependency on `the_repository`.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/fast-import.c | 3 ++-\n builtin/index-pack.c | 2 +-\n object-store.c | 13 -------------\n object-store.h | 7 -------\n path.c | 14 ++++++++++++++\n path.h | 7 +++++++\n 6 files changed, 24 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin/fast-import.c b/builtin/fast-import.c\nindex c1e198f4e34..b2839c5f439 100644\n--- a/builtin/fast-import.c\n+++ b/builtin/fast-import.c\n@@ -811,7 +811,8 @@ static char *keep_pack(const char *curr_index_name)\n \tint keep_fd;\n \n \todb_pack_name(pack_data->repo, &name, pack_data->hash, \"keep\");\n-\tkeep_fd = odb_pack_keep(name.buf);\n+\tkeep_fd = safe_create_file_with_leading_directories(pack_data->repo,\n+\t\t\t\t\t\t\t name.buf);\n \tif (keep_fd < 0)\n \t\tdie_errno(\"cannot create keep file\");\n \twrite_or_die(keep_fd, keep_msg, strlen(keep_msg));\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 60a8ee05dbc..f49431d626b 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1565,7 +1565,7 @@ static void write_special_file(const char *suffix, const char *msg,\n \telse\n \t\tfilename = odb_pack_name(the_repository, &name_buf, hash, suffix);\n \n-\tfd = odb_pack_keep(filename);\n+\tfd = safe_create_file_with_leading_directories(the_repository, filename);\n \tif (fd < 0) {\n \t\tif (errno != EEXIST)\n \t\t\tdie_errno(_(\"cannot write %s file '%s'\"),\ndiff --git a/object-store.c b/object-store.c\nindex e5cfb8c0079..0cbad5a19a0 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -83,19 +83,6 @@ int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)\n \treturn xmkstemp_mode(temp_filename->buf, mode);\n }\n \n-int odb_pack_keep(const char *name)\n-{\n-\tint fd;\n-\n-\tfd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n-\tif (0 <= fd)\n-\t\treturn fd;\n-\n-\t/* slow path */\n-\tsafe_create_leading_directories_const(the_repository, name);\n-\treturn open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n-}\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\ndiff --git a/object-store.h b/object-store.h\nindex 5668de62d01..aa8fc63043e 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -189,13 +189,6 @@ void raw_object_store_clear(struct raw_object_store *o);\n */\n int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n \n-/*\n- * Create a pack .keep file named \"name\" (which should generally be the output\n- * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on\n- * error.\n- */\n-int odb_pack_keep(const char *name);\n-\n void *map_loose_object(struct repository *r, const struct object_id *oid,\n \t\t unsigned long *size);\n \ndiff --git a/path.c b/path.c\nindex 4505bb78e8b..3b598b2847f 100644\n--- a/path.c\n+++ b/path.c\n@@ -1011,6 +1011,20 @@ enum scld_error safe_create_leading_directories_const(struct repository *repo,\n \treturn result;\n }\n \n+int safe_create_file_with_leading_directories(struct repository *repo,\n+\t\t\t\t\t const char *path)\n+{\n+\tint fd;\n+\n+\tfd = open(path, O_RDWR|O_CREAT|O_EXCL, 0600);\n+\tif (0 <= fd)\n+\t\treturn fd;\n+\n+\t/* slow path */\n+\tsafe_create_leading_directories_const(repo, path);\n+\treturn open(path, O_RDWR|O_CREAT|O_EXCL, 0600);\n+}\n+\n static int have_same_root(const char *path1, const char *path2)\n {\n \tint is_abs1, is_abs2;\ndiff --git a/path.h b/path.h\nindex fd1a194b060..e67348f2539 100644\n--- a/path.h\n+++ b/path.h\n@@ -266,6 +266,13 @@ enum scld_error safe_create_leading_directories_const(struct repository *repo,\n \t\t\t\t\t\t const char *path);\n enum scld_error safe_create_leading_directories_no_share(char *path);\n \n+/*\n+ * Create a file, potentially creating its leading directories in case they\n+ * don't exist. Returns the return value of the open(3p) call.\n+ */\n+int safe_create_file_with_leading_directories(struct repository *repo,\n+\t\t\t\t\t const char *path);\n+\n # ifdef USE_THE_REPOSITORY_VARIABLE\n # include \"strbuf.h\"\n # include \"repository.h\"\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516561","messageId":"20250423-pks-object-store-cleanups-v1-5-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 05/13] object-store: allow fetching objects via `has_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:15Z","receivedAt":"2025-04-23T07:48:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"We're about to fully remove `repo_has_object_file()` in favor of\n`has_object()` given that the latter has better defaults: it neither\nreloads packfiles by default nor does it fetch any promised objects in\ncase they are missing.\n\nThe latter usecase keeps us from converting a couple of callsites that\ncurrently do fetch objects though. It is not really clear whether _all_\nof those callsites should be fetching objects, but for a subset of them\nit is the desired behaviour indeed.\n\nIntroduce a new flag `HAS_OBJECT_FETCH_PROMISOR` that causes the\nfunction to optionally fetch missing objects which are part of a\npromisor pack.\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.c | 9 ++++++---\n object-store.h | 10 +++++++---\n 2 files changed, 13 insertions(+), 6 deletions(-)\n\ndiff --git a/object-store.c b/object-store.c\nindex 0cbad5a19a0..0d873868a6d 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -937,12 +937,15 @@ void *read_object_with_reference(struct repository *r,\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags)\n {\n-\tint quick = !(flags & HAS_OBJECT_RECHECK_PACKED);\n-\tunsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |\n-\t\t(quick ? OBJECT_INFO_QUICK : 0);\n+\tunsigned object_info_flags = 0;\n \n \tif (!startup_info->have_repository)\n \t\treturn 0;\n+\tif (!(flags & HAS_OBJECT_RECHECK_PACKED))\n+\t\tobject_info_flags |= OBJECT_INFO_QUICK;\n+\tif (!(flags & HAS_OBJECT_FETCH_PROMISOR))\n+\t\tobject_info_flags |= OBJECT_INFO_SKIP_FETCH_OBJECT;\n+\n \treturn oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;\n }\n \ndiff --git a/object-store.h b/object-store.h\nindex 5bbdaba92d1..c6055376f49 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -266,12 +266,16 @@ int oid_object_info_extended(struct repository *r,\n \t\t\t const struct object_id *,\n \t\t\t struct object_info *, unsigned flags);\n \n-/* Retry packed storage after checking packed and loose storage */\n-#define HAS_OBJECT_RECHECK_PACKED 1\n+enum {\n+\t/* Retry packed storage after checking packed and loose storage */\n+\tHAS_OBJECT_RECHECK_PACKED = (1 << 0),\n+\t/* Allow fetching the object in case the repository has a promisor remote. */\n+\tHAS_OBJECT_FETCH_PROMISOR = (1 << 1),\n+};\n \n /*\n * Returns 1 if the object exists. This function will not lazily fetch objects\n- * in a partial clone.\n+ * in a partial clone by default.\n */\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags);\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516562","messageId":"20250423-pks-object-store-cleanups-v1-4-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 04/13] object-store: move function declarations to their respective subsystems","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:14Z","receivedAt":"2025-04-23T07:48:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"We carry declarations for a couple of functions in \"object-store.h\" that\nare not defined in \"object-store.c\", but in a different subsystem. Move\nthese declarations to the respective headers whose matching code files\ncarry the corresponding definition.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/count-objects.c | 2 +-\n builtin/gc.c | 2 +-\n object-file.h | 73 +++++++++++++++++++++++++++++++++++++++\n object-name.c | 2 +-\n object-store.h | 91 +------------------------------------------------\n packfile.h | 19 +++++++++++\n prune-packed.c | 2 +-\n reachable.c | 2 +-\n 8 files changed, 98 insertions(+), 95 deletions(-)\n\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex 0bb5360b2f2..a88c0c9c09a 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -12,7 +12,7 @@\n #include \"parse-options.h\"\n #include \"quote.h\"\n #include \"packfile.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n \n static unsigned long garbage;\n static off_t size_garbage;\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex b5ce1d32766..4d428f3253d 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -28,7 +28,7 @@\n #include \"commit.h\"\n #include \"commit-graph.h\"\n #include \"packfile.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"pack.h\"\n #include \"pack-objects.h\"\n #include \"path.h\"\ndiff --git a/object-file.h b/object-file.h\nindex 0a7b6b9f9d9..de6dd205ed8 100644\n--- a/object-file.h\n+++ b/object-file.h\n@@ -3,6 +3,7 @@\n \n #include \"git-zlib.h\"\n #include \"object.h\"\n+#include \"object-store.h\"\n \n struct index_state;\n \n@@ -25,6 +26,16 @@ int index_path(struct index_state *istate, struct object_id *oid, const char *pa\n \n struct object_directory;\n \n+/*\n+ * Populate and return the loose object cache array corresponding to the\n+ * given object ID.\n+ */\n+struct oidtree *odb_loose_cache(struct object_directory *odb,\n+\t\t\t\tconst struct object_id *oid);\n+\n+/* Empty the loose object cache for the specified object directory. */\n+void odb_clear_loose_cache(struct object_directory *odb);\n+\n /*\n * Put in `buf` the name of the file in the local object database that\n * would be used to store a loose object with the specified oid.\n@@ -42,6 +53,68 @@ int has_loose_object_nonlocal(const struct object_id *);\n \n int has_loose_object(const struct object_id *);\n \n+void *map_loose_object(struct repository *r, const struct object_id *oid,\n+\t\t unsigned long *size);\n+\n+/*\n+ * Iterate over the files in the loose-object parts of the object\n+ * directory \"path\", triggering the following callbacks:\n+ *\n+ * - loose_object is called for each loose object we find.\n+ *\n+ * - loose_cruft is called for any files that do not appear to be\n+ * loose objects. Note that we only look in the loose object\n+ * directories \"objects/[0-9a-f]{2}/\", so we will not report\n+ * \"objects/foobar\" as cruft.\n+ *\n+ * - loose_subdir is called for each top-level hashed subdirectory\n+ * of the object directory (e.g., \"$OBJDIR/f0\"). It is called\n+ * after the objects in the directory are processed.\n+ *\n+ * Any callback that is NULL will be ignored. Callbacks returning non-zero\n+ * will end the iteration.\n+ *\n+ * In the \"buf\" variant, \"path\" is a strbuf which will also be used as a\n+ * scratch buffer, but restored to its original contents before\n+ * the function returns.\n+ */\n+typedef int each_loose_object_fn(const struct object_id *oid,\n+\t\t\t\t const char *path,\n+\t\t\t\t void *data);\n+typedef int each_loose_cruft_fn(const char *basename,\n+\t\t\t\tconst char *path,\n+\t\t\t\tvoid *data);\n+typedef int each_loose_subdir_fn(unsigned int nr,\n+\t\t\t\t const char *path,\n+\t\t\t\t void *data);\n+int for_each_file_in_obj_subdir(unsigned int subdir_nr,\n+\t\t\t\tstruct strbuf *path,\n+\t\t\t\teach_loose_object_fn obj_cb,\n+\t\t\t\teach_loose_cruft_fn cruft_cb,\n+\t\t\t\teach_loose_subdir_fn subdir_cb,\n+\t\t\t\tvoid *data);\n+int for_each_loose_file_in_objdir(const char *path,\n+\t\t\t\t each_loose_object_fn obj_cb,\n+\t\t\t\t each_loose_cruft_fn cruft_cb,\n+\t\t\t\t each_loose_subdir_fn subdir_cb,\n+\t\t\t\t void *data);\n+int for_each_loose_file_in_objdir_buf(struct strbuf *path,\n+\t\t\t\t each_loose_object_fn obj_cb,\n+\t\t\t\t each_loose_cruft_fn cruft_cb,\n+\t\t\t\t each_loose_subdir_fn subdir_cb,\n+\t\t\t\t void *data);\n+\n+/*\n+ * Iterate over all accessible loose objects without respect to\n+ * reachability. By default, this includes both local and alternate objects.\n+ * The order in which objects are visited is unspecified.\n+ *\n+ * Any flags specific to packs are ignored.\n+ */\n+int for_each_loose_object(each_loose_object_fn, void *,\n+\t\t\t enum for_each_object_flags flags);\n+\n+\n /**\n * format_object_header() is a thin wrapper around s xsnprintf() that\n * writes the initial \" \" part of the loose object\ndiff --git a/object-name.c b/object-name.c\nindex 2c751a5352a..9288b2dd245 100644\n--- a/object-name.c\n+++ b/object-name.c\n@@ -19,7 +19,7 @@\n #include \"oidtree.h\"\n #include \"packfile.h\"\n #include \"pretty.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"read-cache-ll.h\"\n #include \"repo-settings.h\"\n #include \"repository.h\"\ndiff --git a/object-store.h b/object-store.h\nindex aa8fc63043e..5bbdaba92d1 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -82,16 +82,6 @@ struct object_directory *set_temporary_primary_odb(const char *dir, int will_des\n */\n void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);\n \n-/*\n- * Populate and return the loose object cache array corresponding to the\n- * given object ID.\n- */\n-struct oidtree *odb_loose_cache(struct object_directory *odb,\n-\t\t\t\t const struct object_id *oid);\n-\n-/* Empty the loose object cache for the specified object directory. */\n-void odb_clear_loose_cache(struct object_directory *odb);\n-\n struct packed_git;\n struct multi_pack_index;\n struct cached_object_entry;\n@@ -189,9 +179,6 @@ void raw_object_store_clear(struct raw_object_store *o);\n */\n int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n \n-void *map_loose_object(struct repository *r, const struct object_id *oid,\n-\t\t unsigned long *size);\n-\n void *repo_read_object_file(struct repository *r,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type *type,\n@@ -340,56 +327,7 @@ static inline void obj_read_unlock(void)\n \tif(obj_read_use_lock)\n \t\tpthread_mutex_unlock(&obj_read_mutex);\n }\n-\n-/*\n- * Iterate over the files in the loose-object parts of the object\n- * directory \"path\", triggering the following callbacks:\n- *\n- * - loose_object is called for each loose object we find.\n- *\n- * - loose_cruft is called for any files that do not appear to be\n- * loose objects. Note that we only look in the loose object\n- * directories \"objects/[0-9a-f]{2}/\", so we will not report\n- * \"objects/foobar\" as cruft.\n- *\n- * - loose_subdir is called for each top-level hashed subdirectory\n- * of the object directory (e.g., \"$OBJDIR/f0\"). It is called\n- * after the objects in the directory are processed.\n- *\n- * Any callback that is NULL will be ignored. Callbacks returning non-zero\n- * will end the iteration.\n- *\n- * In the \"buf\" variant, \"path\" is a strbuf which will also be used as a\n- * scratch buffer, but restored to its original contents before\n- * the function returns.\n- */\n-typedef int each_loose_object_fn(const struct object_id *oid,\n-\t\t\t\t const char *path,\n-\t\t\t\t void *data);\n-typedef int each_loose_cruft_fn(const char *basename,\n-\t\t\t\tconst char *path,\n-\t\t\t\tvoid *data);\n-typedef int each_loose_subdir_fn(unsigned int nr,\n-\t\t\t\t const char *path,\n-\t\t\t\t void *data);\n-int for_each_file_in_obj_subdir(unsigned int subdir_nr,\n-\t\t\t\tstruct strbuf *path,\n-\t\t\t\teach_loose_object_fn obj_cb,\n-\t\t\t\teach_loose_cruft_fn cruft_cb,\n-\t\t\t\teach_loose_subdir_fn subdir_cb,\n-\t\t\t\tvoid *data);\n-int for_each_loose_file_in_objdir(const char *path,\n-\t\t\t\t each_loose_object_fn obj_cb,\n-\t\t\t\t each_loose_cruft_fn cruft_cb,\n-\t\t\t\t each_loose_subdir_fn subdir_cb,\n-\t\t\t\t void *data);\n-int for_each_loose_file_in_objdir_buf(struct strbuf *path,\n-\t\t\t\t each_loose_object_fn obj_cb,\n-\t\t\t\t each_loose_cruft_fn cruft_cb,\n-\t\t\t\t each_loose_subdir_fn subdir_cb,\n-\t\t\t\t void *data);\n-\n-/* Flags for for_each_*_object() below. */\n+/* Flags for for_each_*_object(). */\n enum for_each_object_flags {\n \t/* Iterate only over local objects, not alternates. */\n \tFOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),\n@@ -409,33 +347,6 @@ enum for_each_object_flags {\n \tFOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),\n };\n \n-/*\n- * Iterate over all accessible loose objects without respect to\n- * reachability. By default, this includes both local and alternate objects.\n- * The order in which objects are visited is unspecified.\n- *\n- * Any flags specific to packs are ignored.\n- */\n-int for_each_loose_object(each_loose_object_fn, void *,\n-\t\t\t enum for_each_object_flags flags);\n-\n-/*\n- * Iterate over all accessible packed objects without respect to reachability.\n- * By default, this includes both local and alternate packs.\n- *\n- * Note that some objects may appear twice if they are found in multiple packs.\n- * Each pack is visited in an unspecified order. By default, objects within a\n- * pack are visited in pack-idx order (i.e., sorted by oid).\n- */\n-typedef int each_packed_object_fn(const struct object_id *oid,\n-\t\t\t\t struct packed_git *pack,\n-\t\t\t\t uint32_t pos,\n-\t\t\t\t void *data);\n-int for_each_object_in_pack(struct packed_git *p,\n-\t\t\t each_packed_object_fn, void *data,\n-\t\t\t enum for_each_object_flags flags);\n-int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,\n-\t\t\t void *data, enum for_each_object_flags flags);\n \n void *read_object_with_reference(struct repository *r,\n \t\t\t\t const struct object_id *oid,\ndiff --git a/packfile.h b/packfile.h\nindex 05499382397..3a3c77cf05a 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -3,6 +3,7 @@\n \n #include \"list.h\"\n #include \"object.h\"\n+#include \"object-store.h\"\n #include \"oidset.h\"\n \n /* in object-store.h */\n@@ -117,6 +118,24 @@ void for_each_file_in_pack_dir(const char *objdir,\n \t\t\t each_file_in_pack_dir_fn fn,\n \t\t\t void *data);\n \n+/*\n+ * Iterate over all accessible packed objects without respect to reachability.\n+ * By default, this includes both local and alternate packs.\n+ *\n+ * Note that some objects may appear twice if they are found in multiple packs.\n+ * Each pack is visited in an unspecified order. By default, objects within a\n+ * pack are visited in pack-idx order (i.e., sorted by oid).\n+ */\n+typedef int each_packed_object_fn(const struct object_id *oid,\n+\t\t\t\t struct packed_git *pack,\n+\t\t\t\t uint32_t pos,\n+\t\t\t\t void *data);\n+int for_each_object_in_pack(struct packed_git *p,\n+\t\t\t each_packed_object_fn, void *data,\n+\t\t\t enum for_each_object_flags flags);\n+int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,\n+\t\t\t void *data, enum for_each_object_flags flags);\n+\n /* A hook to report invalid files in pack directory */\n #define PACKDIR_FILE_PACK 1\n #define PACKDIR_FILE_IDX 2\ndiff --git a/prune-packed.c b/prune-packed.c\nindex c1d95a519d7..92fb4fbb0ed 100644\n--- a/prune-packed.c\n+++ b/prune-packed.c\n@@ -2,7 +2,7 @@\n \n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"packfile.h\"\n #include \"progress.h\"\n #include \"prune-packed.h\"\ndiff --git a/reachable.c b/reachable.c\nindex e5f56f40181..9dc748f0b9a 100644\n--- a/reachable.c\n+++ b/reachable.c\n@@ -14,7 +14,7 @@\n #include \"list-objects.h\"\n #include \"packfile.h\"\n #include \"worktree.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"pack-bitmap.h\"\n #include \"pack-mtimes.h\"\n #include \"config.h\"\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516565","messageId":"20250423-pks-object-store-cleanups-v1-6-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 06/13] treewide: trivial conversions of `repo_has_object_file()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:16Z","receivedAt":"2025-04-23T07:48:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"As the comment of `repo_has_object_file()` and its `_with_flags()`\nvariant tells us, these functions are considered to be deprecated in\nfavor of `has_object()`. The benefit of the replacement is better\ndefaults: it doesn't fetch missing objects via promisor remotes, and\nneither does it reload packfiles if an object wasn't found by default.\n\nStart sunsetting the functions by replacing trivial callsites of it with\n`has_object()`:\n\n - `repo_has_object_file(...)` is equivalent to\n `has_object(..., HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)`.\n\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT)`\n is equivalent to `has_object(..., 0)`.\n\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_SKIP_FETCH_OBJECT)`\n is equivalent to `has_object(..., HAS_OBJECT_RECHECK_PACKED)`.\n\nWhile converting callsites to use `HAS_OBJECT_FETCH_PROMISOR` retains\nthe current behaviour, it is not entirely clear whether all of them\nreally should be fetching promised objects. A subset of such callsites\nwhere it is obvious that they really shouldn't fetch objects has been\nleft out of this commit and will be adapted over the next couple of\ncommits.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/cat-file.c | 3 ++-\n builtin/clone.c | 4 +---\n builtin/fetch.c | 15 +++++++--------\n builtin/receive-pack.c | 4 +++-\n builtin/remote.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n cache-tree.c | 13 +++++++++----\n fetch-pack.c | 7 +++----\n http-push.c | 11 +++++++----\n notes.c | 3 ++-\n object-store.c | 2 +-\n reflog.c | 3 ++-\n remote.c | 2 +-\n send-pack.c | 5 +----\n shallow.c | 9 ++++++---\n upload-pack.c | 3 +--\n walker.c | 3 ++-\n 17 files changed, 52 insertions(+), 41 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 0e3f10a9467..3914a2a3f61 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -169,7 +169,8 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name,\n \t\tgoto cleanup;\n \n \tcase 'e':\n-\t\tret = !repo_has_object_file(the_repository, &oid);\n+\t\tret = !has_object(the_repository, &oid,\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR);\n \t\tgoto cleanup;\n \n \tcase 'w':\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 6b1d11a3ed2..b498b81a043 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -504,9 +504,7 @@ static void write_followtags(const struct ref *refs, const char *msg)\n \t\t\tcontinue;\n \t\tif (ends_with(ref->name, \"^{}\"))\n \t\t\tcontinue;\n-\t\tif (!repo_has_object_file_with_flags(the_repository, &ref->old_oid,\n-\t\t\t\t\t\t OBJECT_INFO_QUICK |\n-\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (!has_object(the_repository, &ref->old_oid, 0))\n \t\t\tcontinue;\n \t\trefs_update_ref(get_main_ref_store(the_repository), msg,\n \t\t\t\tref->name, &ref->old_oid, NULL, 0,\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 95589b49948..aadcf49a5b4 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -337,7 +337,6 @@ static void find_non_local_tags(const struct ref *refs,\n \tstruct string_list_item *remote_ref_item;\n \tconst struct ref *ref;\n \tstruct refname_hash_entry *item = NULL;\n-\tconst int quick_flags = OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT;\n \n \trefname_hash_init(&existing_refs);\n \trefname_hash_init(&remote_refs);\n@@ -367,9 +366,9 @@ static void find_non_local_tags(const struct ref *refs,\n \t\t */\n \t\tif (ends_with(ref->name, \"^{}\")) {\n \t\t\tif (item &&\n-\t\t\t !repo_has_object_file_with_flags(the_repository, &ref->old_oid, quick_flags) &&\n+\t\t\t !has_object(the_repository, &ref->old_oid, 0) &&\n \t\t\t !oidset_contains(&fetch_oids, &ref->old_oid) &&\n-\t\t\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t\t\t !has_object(the_repository, &item->oid, 0) &&\n \t\t\t !oidset_contains(&fetch_oids, &item->oid))\n \t\t\t\tclear_item(item);\n \t\t\titem = NULL;\n@@ -383,7 +382,7 @@ static void find_non_local_tags(const struct ref *refs,\n \t\t * fetch.\n \t\t */\n \t\tif (item &&\n-\t\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t\t !has_object(the_repository, &item->oid, 0) &&\n \t\t !oidset_contains(&fetch_oids, &item->oid))\n \t\t\tclear_item(item);\n \n@@ -404,7 +403,7 @@ static void find_non_local_tags(const struct ref *refs,\n \t * checked to see if it needs fetching.\n \t */\n \tif (item &&\n-\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t !has_object(the_repository, &item->oid, 0) &&\n \t !oidset_contains(&fetch_oids, &item->oid))\n \t\tclear_item(item);\n \n@@ -911,7 +910,8 @@ static int update_local_ref(struct ref *ref,\n \tstruct commit *current = NULL, *updated;\n \tint fast_forward = 0;\n \n-\tif (!repo_has_object_file(the_repository, &ref->new_oid))\n+\tif (!has_object(the_repository, &ref->new_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\tdie(_(\"object %s not found\"), oid_to_hex(&ref->new_oid));\n \n \tif (oideq(&ref->old_oid, &ref->new_oid)) {\n@@ -1330,8 +1330,7 @@ static int check_exist_and_connected(struct ref *ref_map)\n \t * we need all direct targets to exist.\n \t */\n \tfor (r = rm; r; r = r->next) {\n-\t\tif (!repo_has_object_file_with_flags(the_repository, &r->old_oid,\n-\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (!has_object(the_repository, &r->old_oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\treturn -1;\n \t}\n \ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex be314879e82..c92e57ba188 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -1506,7 +1506,9 @@ static const char *update(struct command *cmd, struct shallow_info *si)\n \t\t}\n \t}\n \n-\tif (!is_null_oid(new_oid) && !repo_has_object_file(the_repository, new_oid)) {\n+\tif (!is_null_oid(new_oid) &&\n+\t !has_object(the_repository, new_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\terror(\"unpack should have generated %s, \"\n \t\t \"but I can't find it!\", oid_to_hex(new_oid));\n \t\tret = \"bad pack\";\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex b4baa34e665..0d6755bcb71 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -454,7 +454,8 @@ static int get_push_ref_states(const struct ref *remote_refs,\n \t\t\tinfo->status = PUSH_STATUS_UPTODATE;\n \t\telse if (is_null_oid(&ref->old_oid))\n \t\t\tinfo->status = PUSH_STATUS_CREATE;\n-\t\telse if (repo_has_object_file(the_repository, &ref->old_oid) &&\n+\t\telse if (has_object(the_repository, &ref->old_oid,\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) &&\n \t\t\t ref_newer(&ref->new_oid, &ref->old_oid))\n \t\t\tinfo->status = PUSH_STATUS_FASTFORWARD;\n \t\telse\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex 661be789f13..e905d5f4e19 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -449,7 +449,8 @@ static void unpack_delta_entry(enum object_type type, unsigned long delta_size,\n \t\tdelta_data = get_data(delta_size);\n \t\tif (!delta_data)\n \t\t\treturn;\n-\t\tif (repo_has_object_file(the_repository, &base_oid))\n+\t\tif (has_object(the_repository, &base_oid,\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\t; /* Ok we have this one */\n \t\telse if (resolve_against_held(nr, &base_oid,\n \t\t\t\t\t delta_data, delta_size))\ndiff --git a/cache-tree.c b/cache-tree.c\nindex c0e1e9ee1d4..fa3858e2829 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -238,7 +238,9 @@ int cache_tree_fully_valid(struct cache_tree *it)\n \tint i;\n \tif (!it)\n \t\treturn 0;\n-\tif (it->entry_count < 0 || !repo_has_object_file(the_repository, &it->oid))\n+\tif (it->entry_count < 0 ||\n+\t has_object(the_repository, &it->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn 0;\n \tfor (i = 0; i < it->subtree_nr; i++) {\n \t\tif (!cache_tree_fully_valid(it->down[i]->cache_tree))\n@@ -289,7 +291,9 @@ static int update_one(struct cache_tree *it,\n \t\t}\n \t}\n \n-\tif (0 <= it->entry_count && repo_has_object_file(the_repository, &it->oid))\n+\tif (0 <= it->entry_count &&\n+\t has_object(the_repository, &it->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn it->entry_count;\n \n \t/*\n@@ -395,7 +399,8 @@ static int update_one(struct cache_tree *it,\n \t\tce_missing_ok = mode == S_IFGITLINK || missing_ok ||\n \t\t\t!must_check_existence(ce);\n \t\tif (is_null_oid(oid) ||\n-\t\t (!ce_missing_ok && !repo_has_object_file(the_repository, oid))) {\n+\t\t (!ce_missing_ok && !has_object(the_repository, oid,\n+\t\t\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))) {\n \t\t\tstrbuf_release(&buffer);\n \t\t\tif (expected_missing)\n \t\t\t\treturn -1;\n@@ -443,7 +448,7 @@ static int update_one(struct cache_tree *it,\n \t\tstruct object_id oid;\n \t\thash_object_file(the_hash_algo, buffer.buf, buffer.len,\n \t\t\t\t OBJ_TREE, &oid);\n-\t\tif (repo_has_object_file_with_flags(the_repository, &oid, OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (has_object(the_repository, &oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\toidcpy(&it->oid, &oid);\n \t\telse\n \t\t\tto_invalidate = 1;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 210dc30d50f..fa4231fee74 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -769,9 +769,7 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator,\n \t\tif (!commit) {\n \t\t\tstruct object *o;\n \n-\t\t\tif (!repo_has_object_file_with_flags(the_repository, &ref->old_oid,\n-\t\t\t\t\t\t\t OBJECT_INFO_QUICK |\n-\t\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\t\tif (!has_object(the_repository, &ref->old_oid, 0))\n \t\t\t\tcontinue;\n \t\t\to = parse_object(the_repository, &ref->old_oid);\n \t\t\tif (!o || o->type != OBJ_COMMIT)\n@@ -1985,7 +1983,8 @@ static void update_shallow(struct fetch_pack_args *args,\n \t\tstruct oid_array extra = OID_ARRAY_INIT;\n \t\tstruct object_id *oid = si->shallow->oid;\n \t\tfor (i = 0; i < si->shallow->nr; i++)\n-\t\t\tif (repo_has_object_file(the_repository, &oid[i]))\n+\t\t\tif (has_object(the_repository, &oid[i],\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\t\toid_array_append(&extra, &oid[i]);\n \t\tif (extra.nr) {\n \t\t\tsetup_alternate_shallow(&shallow_lock,\ndiff --git a/http-push.c b/http-push.c\nindex 32e37565f4e..f9e67cabd4b 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -1446,7 +1446,9 @@ static void one_remote_ref(const char *refname)\n \t * Fetch a copy of the object if it doesn't exist locally - it\n \t * may be required for updating server info later.\n \t */\n-\tif (repo->can_update_info_refs && !repo_has_object_file(the_repository, &ref->old_oid)) {\n+\tif (repo->can_update_info_refs &&\n+\t !has_object(the_repository, &ref->old_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\tobj = lookup_unknown_object(the_repository, &ref->old_oid);\n \t\tfprintf(stderr,\t\" fetch %s for %s\\n\",\n \t\t\toid_to_hex(&ref->old_oid), refname);\n@@ -1651,14 +1653,14 @@ static int delete_remote_branch(const char *pattern, int force)\n \t\t\treturn error(\"Remote HEAD symrefs too deep\");\n \t\tif (is_null_oid(&head_oid))\n \t\t\treturn error(\"Unable to resolve remote HEAD\");\n-\t\tif (!repo_has_object_file(the_repository, &head_oid))\n+\t\tif (!has_object(the_repository, &head_oid, HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn error(\"Remote HEAD resolves to object %s\\nwhich does not exist locally, perhaps you need to fetch?\", oid_to_hex(&head_oid));\n \n \t\t/* Remote branch must resolve to a known object */\n \t\tif (is_null_oid(&remote_ref->old_oid))\n \t\t\treturn error(\"Unable to resolve remote branch %s\",\n \t\t\t\t remote_ref->name);\n-\t\tif (!repo_has_object_file(the_repository, &remote_ref->old_oid))\n+\t\tif (!has_object(the_repository, &remote_ref->old_oid, HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn error(\"Remote branch %s resolves to object %s\\nwhich does not exist locally, perhaps you need to fetch?\", remote_ref->name, oid_to_hex(&remote_ref->old_oid));\n \n \t\t/* Remote branch must be an ancestor of remote HEAD */\n@@ -1879,7 +1881,8 @@ int cmd_main(int argc, const char **argv)\n \t\tif (!force_all &&\n \t\t !is_null_oid(&ref->old_oid) &&\n \t\t !ref->force) {\n-\t\t\tif (!repo_has_object_file(the_repository, &ref->old_oid) ||\n+\t\t\tif (!has_object(the_repository, &ref->old_oid,\n+\t\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) ||\n \t\t\t !ref_newer(&ref->peer_ref->new_oid,\n \t\t\t\t &ref->old_oid)) {\n \t\t\t\t/*\ndiff --git a/notes.c b/notes.c\nindex d9645c4b5dc..0a128f1de98 100644\n--- a/notes.c\n+++ b/notes.c\n@@ -794,7 +794,8 @@ static int prune_notes_helper(const struct object_id *object_oid,\n \tstruct note_delete_list **l = (struct note_delete_list **) cb_data;\n \tstruct note_delete_list *n;\n \n-\tif (repo_has_object_file(the_repository, object_oid))\n+\tif (has_object(the_repository, object_oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn 0; /* nothing to do for this note */\n \n \t/* failed to find object => prune this note */\ndiff --git a/object-store.c b/object-store.c\nindex 0d873868a6d..2db34804e8f 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -847,7 +847,7 @@ int pretend_object_file(struct repository *repo,\n \tchar *co_buf;\n \n \thash_object_file(repo->hash_algo, buf, len, type, oid);\n-\tif (repo_has_object_file_with_flags(repo, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||\n+\tif (has_object(repo, oid, 0) ||\n \t find_cached_object(repo->objects, oid))\n \t\treturn 0;\n \ndiff --git a/reflog.c b/reflog.c\nindex c6c534dfad4..2db1cc81d01 100644\n--- a/reflog.c\n+++ b/reflog.c\n@@ -152,7 +152,8 @@ static int tree_is_complete(const struct object_id *oid)\n \tinit_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size);\n \tcomplete = 1;\n \twhile (tree_entry(&desc, &entry)) {\n-\t\tif (!repo_has_object_file(the_repository, &entry.oid) ||\n+\t\tif (!has_object(the_repository, &entry.oid,\n+\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) ||\n \t\t (S_ISDIR(entry.mode) && !tree_is_complete(&entry.oid))) {\n \t\t\ttree->object.flags |= INCOMPLETE;\n \t\t\tcomplete = 0;\ndiff --git a/remote.c b/remote.c\nindex 9fa3614e7a3..4099183cacd 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -1702,7 +1702,7 @@ void set_ref_status_for_push(struct ref *remote_refs, int send_mirror,\n \t\tif (!reject_reason && !ref->deletion && !is_null_oid(&ref->old_oid)) {\n \t\t\tif (starts_with(ref->name, \"refs/tags/\"))\n \t\t\t\treject_reason = REF_STATUS_REJECT_ALREADY_EXISTS;\n-\t\t\telse if (!repo_has_object_file_with_flags(the_repository, &ref->old_oid, OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\t\telse if (!has_object(the_repository, &ref->old_oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\t\treject_reason = REF_STATUS_REJECT_FETCH_FIRST;\n \t\t\telse if (!lookup_commit_reference_gently(the_repository, &ref->old_oid, 1) ||\n \t\t\t\t !lookup_commit_reference_gently(the_repository, &ref->new_oid, 1))\ndiff --git a/send-pack.c b/send-pack.c\nindex 5005689cb55..86592ce526d 100644\n--- a/send-pack.c\n+++ b/send-pack.c\n@@ -45,10 +45,7 @@ int option_parse_push_signed(const struct option *opt,\n static void feed_object(struct repository *r,\n \t\t\tconst struct object_id *oid, FILE *fh, int negative)\n {\n-\tif (negative &&\n-\t !repo_has_object_file_with_flags(r, oid,\n-\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT |\n-\t\t\t\t\t OBJECT_INFO_QUICK))\n+\tif (negative && !has_object(r, oid, 0))\n \t\treturn;\n \n \tif (negative)\ndiff --git a/shallow.c b/shallow.c\nindex 2f82ebd6e3f..faeeeb45f98 100644\n--- a/shallow.c\n+++ b/shallow.c\n@@ -310,7 +310,8 @@ static int write_one_shallow(const struct commit_graft *graft, void *cb_data)\n \tif (graft->nr_parent != -1)\n \t\treturn 0;\n \tif (data->flags & QUICK) {\n-\t\tif (!repo_has_object_file(the_repository, &graft->oid))\n+\t\tif (!has_object(the_repository, &graft->oid,\n+\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn 0;\n \t} else if (data->flags & SEEN_ONLY) {\n \t\tstruct commit *c = lookup_commit(the_repository, &graft->oid);\n@@ -476,7 +477,8 @@ void prepare_shallow_info(struct shallow_info *info, struct oid_array *sa)\n \tALLOC_ARRAY(info->ours, sa->nr);\n \tALLOC_ARRAY(info->theirs, sa->nr);\n \tfor (size_t i = 0; i < sa->nr; i++) {\n-\t\tif (repo_has_object_file(the_repository, sa->oid + i)) {\n+\t\tif (has_object(the_repository, sa->oid + i,\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\t\tstruct commit_graft *graft;\n \t\t\tgraft = lookup_commit_graft(the_repository,\n \t\t\t\t\t\t &sa->oid[i]);\n@@ -513,7 +515,8 @@ void remove_nonexistent_theirs_shallow(struct shallow_info *info)\n \tfor (i = dst = 0; i < info->nr_theirs; i++) {\n \t\tif (i != dst)\n \t\t\tinfo->theirs[dst] = info->theirs[i];\n-\t\tif (repo_has_object_file(the_repository, oid + info->theirs[i]))\n+\t\tif (has_object(the_repository, oid + info->theirs[i],\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\tdst++;\n \t}\n \tinfo->nr_theirs = dst;\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 30e4630f3a1..956da5b061a 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -509,8 +509,7 @@ static int got_oid(struct upload_pack_data *data,\n {\n \tif (get_oid_hex(hex, oid))\n \t\tdie(\"git upload-pack: expected SHA1 object, got '%s'\", hex);\n-\tif (!repo_has_object_file_with_flags(the_repository, oid,\n-\t\t\t\t\t OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\tif (!has_object(the_repository, oid, 0))\n \t\treturn -1;\n \treturn do_got_oid(data, oid);\n }\ndiff --git a/walker.c b/walker.c\nindex 4fedc19f346..b470d43e54d 100644\n--- a/walker.c\n+++ b/walker.c\n@@ -150,7 +150,8 @@ static int process(struct walker *walker, struct object *obj)\n \t\treturn 0;\n \tobj->flags |= SEEN;\n \n-\tif (repo_has_object_file(the_repository, &obj->oid)) {\n+\tif (has_object(the_repository, &obj->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\t/* We already have it, so we should scan it now. */\n \t\tobj->flags |= TO_SCAN;\n \t}\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516563","messageId":"20250423-pks-object-store-cleanups-v1-7-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 07/13] builtin/index-pack: don't fetch promised objects for collision check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:17Z","receivedAt":"2025-04-23T07:48:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Any packed objects indexed via git-index-pack(1) are subject to a\ncollision check. This collision check has the intent to determine\nwhether we already have an object with the same object ID, but different\ncontents in the repository.\n\nThe check whether the collision check is really needed is performed via\n`repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)`. But unless\nexplicitly told otherwise via `OBJECT_INFO_SKIP_FETCH_OBJECT`, this\nfunction will also cause us to fetch the object ID in case it is part of\na promisor pack. As such, we may end up fetching the object only to\ncheck whether the fetched object and the object that we're indexing have\nthe same content.\n\nThis behaviour is highly dubious and more likely than not unintended.\nFix it by converting to `has_object()`, which knows to neither reload\npackfiles nor to fetch promisor objects by default.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/index-pack.c | 4 +---\n 1 file changed, 1 insertion(+), 3 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex f49431d626b..805b7aa1e28 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -892,9 +892,7 @@ static void sha1_object(const void *data, struct object_entry *obj_entry,\n \n \tif (startup_info->have_repository) {\n \t\tread_lock();\n-\t\tcollision_test_needed =\n-\t\t\trepo_has_object_file_with_flags(the_repository, oid,\n-\t\t\t\t\t\t\tOBJECT_INFO_QUICK);\n+\t\tcollision_test_needed = has_object(the_repository, oid, 0);\n \t\tread_unlock();\n \t}\n \n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516564","messageId":"20250423-pks-object-store-cleanups-v1-8-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 08/13] builtin/show-ref: don't fetch objects when printing refs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:18Z","receivedAt":"2025-04-23T07:48:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When printing references via git-show-ref(1) we double check that their\nrespective object IDs point to an existing object. This check is\nperformed via `repo_has_object_file()`, which knows to fetch missing\npromised objects in the background. We shouldn't have a need to fetch\nsuch objects though as no reference should ever point to a missing\nobject at all.\n\nConvert the callsite to `has_object()`, which doesn't fetch promisor\nobjects by default.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/show-ref.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/builtin/show-ref.c b/builtin/show-ref.c\nindex f81209f23c3..fe592b4c202 100644\n--- a/builtin/show-ref.c\n+++ b/builtin/show-ref.c\n@@ -35,7 +35,7 @@ static void show_one(const struct show_one_options *opts,\n \tconst char *hex;\n \tstruct object_id peeled;\n \n-\tif (!repo_has_object_file(the_repository, oid))\n+\tif (!has_object(the_repository, oid, HAS_OBJECT_RECHECK_PACKED))\n \t\tdie(\"git show-ref: bad ref %s (%s)\", refname,\n \t\t oid_to_hex(oid));\n \n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516566","messageId":"20250423-pks-object-store-cleanups-v1-9-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 09/13] refs: don't fetch promisor objects in `ref_resolves_to_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:19Z","receivedAt":"2025-04-23T07:48:32Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Similar to the preceding commit, don't try to fetch objects pointed to\nby references. Any reference whose object does not exist is broken by\ndefinition an, so we should report it accordingly.\n\nSigned-off-by: Patrick Steinhardt \n---\n refs.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 6559db37890..0492cf0d13a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -376,7 +376,7 @@ int ref_resolves_to_object(const char *refname,\n {\n \tif (flags & REF_ISBROKEN)\n \t\treturn 0;\n-\tif (!repo_has_object_file(repo, oid)) {\n+\tif (!has_object(repo, oid, HAS_OBJECT_RECHECK_PACKED)) {\n \t\terror(_(\"%s does not point to a valid object!\"), refname);\n \t\treturn 0;\n \t}\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516567","messageId":"20250423-pks-object-store-cleanups-v1-10-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 10/13] http-walker: don't fetch objects via promisor remotes","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:20Z","receivedAt":"2025-04-23T07:48:34Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The HTTP walker is responsible for fetching objects via the dumb HTTP\nprotocol. To avoid re-fetching objects that we already have locally we\nfirst check whether the object already exists in the local repository\nbefore issuing the requests.\n\nThis existence check is done by calling `repo_has_object_file()`, which\nwill fetch the object via a promisor remote in case it is in a promisor\npack. This fetch does not make any sense for us though: we're already in\nthe process of fetching the object anyway, so fetching it via a separate\nconnection is wasteful, but should otherwise be harmless.\n\nFix the issue by converting to `has_object()`, which knows to not fetch\nobjects via promisor remotes by default.\n\nSigned-off-by: Patrick Steinhardt \n---\n http-walker.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/http-walker.c b/http-walker.c\nindex 95458e2f638..5ad2eae9a11 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -138,7 +138,7 @@ static int fill_active_slot(void *data UNUSED)\n \tlist_for_each_safe(pos, tmp, head) {\n \t\tobj_req = list_entry(pos, struct object_request, node);\n \t\tif (obj_req->state == WAITING) {\n-\t\t\tif (repo_has_object_file(the_repository, &obj_req->oid))\n+\t\t\tif (has_object(the_repository, &obj_req->oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\t\tobj_req->state = COMPLETE;\n \t\t\telse {\n \t\t\t\tstart_object_request(obj_req);\n@@ -496,7 +496,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)\n \tif (!obj_req)\n \t\treturn error(\"Couldn't find request for %s in the queue\", hex);\n \n-\tif (repo_has_object_file(the_repository, &obj_req->oid)) {\n+\tif (has_object(the_repository, &obj_req->oid, HAS_OBJECT_RECHECK_PACKED)) {\n \t\tif (obj_req->req)\n \t\t\tabort_http_object_request(&obj_req->req);\n \t\tabort_object_request(obj_req);\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516568","messageId":"20250423-pks-object-store-cleanups-v1-11-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 11/13] list-objects: clarify how promised blobs are excluded","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:21Z","receivedAt":"2025-04-23T07:48:34Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When \"--exclude-promisor-objects\" is passed by the user, commands like\ngit-rev-list(1) will exclude any objects part of a promisor pack. For\nblobs this logic is handled in `process_blob()` with the following\nlogic:\n\n if (ctx->revs->exclude_promisor_objects &&\n !repo_has_object_file(the_repository, &obj->oid) &&\n is_promisor_object(ctx->revs->repo, &obj->oid))\n return;\n\nIt is somewhat puzzling that we use `repo_has_object_file()` to check\nfor existence of the blob because this function will cause us to fetch\nmissing objects in case they are part of a promisor pack. As such, one\nmay wonder whether the logic to exclude promised blobs is completely\nbroken.\n\nAs it turns out it's not broken: when \"--exclude-promisor-objects\" is\nset we also unset the global `fetch_if_missing` variable, which causes\n`do_oid_object_info_extended()` to not fetch any promised objects at\nall.\n\nClarify this logic by using `has_object()`, which doesn't fetch promised\nobjects by default.\n\nSigned-off-by: Patrick Steinhardt \n---\n list-objects.c | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/list-objects.c b/list-objects.c\nindex 1e5512e1318..cae4f7aff8c 100644\n--- a/list-objects.c\n+++ b/list-objects.c\n@@ -74,7 +74,8 @@ static void process_blob(struct traversal_context *ctx,\n \t * of missing objects.\n \t */\n \tif (ctx->revs->exclude_promisor_objects &&\n-\t !repo_has_object_file(the_repository, &obj->oid) &&\n+\t !has_object(the_repository, &obj->oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED) &&\n \t is_promisor_object(ctx->revs->repo, &obj->oid))\n \t\treturn;\n \n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516569","messageId":"20250423-pks-object-store-cleanups-v1-12-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 12/13] bulk-checkin: don't fetch promised objects on write","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:22Z","receivedAt":"2025-04-23T07:48:35Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When writing objects via the bulk-checkin subsystem we first try to\nfigure out whether an object already exists in the repository before we\nappend it to the packfile. This check uses `repo_has_object_file()`,\nwhich knows to fetch promised objects by default. As such, if we were\nabout to write an object that is promised, we'd fetch the object via the\npromisor and then skip writing it. This behaviour doesn't seem sensible:\nit should be significantly faster to take the locally-written object\ninstead of faulting in objects from the promisor remote.\n\nThere is one counter-argument here: it could be that the bulk-checkin\nmechanism will end up writing an object to disk whose content collides\nwith the object in the promisor remote. The local repository and its\npromisor remote would now have two objects with different contents but\nthe same name. But the resulting behaviour would be wrong both when we\nprefer the fetched object, and also when prefering the written object:\n\n - When we prefer the written object we will now see a different world\n compared to everyone else who has the promised object.\n\n - When we prefer the fetched object we will end up with an object that\n is different compared to what the user just asked us to write. This\n seems even worse compared to the first scenario.\n\nIn an ideal world, we would protect against this by fetching the\npromised object and then performing a collision check. But this feels\nexceedingly expensive and ultimately rather pointless, as more common\nwriting paths like `write_loose_object()` don't protect against this\nscenario either. And in any case we're talking about a local user that\nhas write access to the repository anyway, so if they want to do any\nkind of mischieve they already can.\n\nChange the behaviour so that we don't fault in the object via the\npromisor remote. We shouldn't have to worry about hash collisions too\nmuch (yet) as the mechanism is only used during local writes anyway. And\neven if there was a collision, prefering local data that we were just\nasked to write over data controlled by a potentially untrusted remote\nfeels like the better failure mode.\n\nSigned-off-by: Patrick Steinhardt \n---\n bulk-checkin.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/bulk-checkin.c b/bulk-checkin.c\nindex c31c31b18d8..b182c456d69 100644\n--- a/bulk-checkin.c\n+++ b/bulk-checkin.c\n@@ -130,7 +130,7 @@ static void flush_batch_fsync(void)\n static int already_written(struct bulk_checkin_packfile *state, struct object_id *oid)\n {\n \t/* The object may already exist in the repository */\n-\tif (repo_has_object_file(the_repository, oid))\n+\tif (has_object(the_repository, oid, HAS_OBJECT_RECHECK_PACKED))\n \t\treturn 1;\n \n \t/* Might want to keep the list sorted */\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516570","messageId":"20250423-pks-object-store-cleanups-v1-13-81f8411a5d08@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH 13/13] object-store: drop `repo_has_object_file()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-23T07:48:23Z","receivedAt":"2025-04-23T07:48:36Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In the preceding commits we have converted all users of\n`repo_has_object_file()` and its `_with_flags()` variant to instead use\n`has_object()`. Drop these functions.\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.c | 14 --------------\n object-store.h | 17 -----------------\n 2 files changed, 31 deletions(-)\n\ndiff --git a/object-store.c b/object-store.c\nindex 2db34804e8f..2f51d0e3b03 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -949,20 +949,6 @@ int has_object(struct repository *r, const struct object_id *oid,\n \treturn oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;\n }\n \n-int repo_has_object_file_with_flags(struct repository *r,\n-\t\t\t\t const struct object_id *oid, int flags)\n-{\n-\tif (!startup_info->have_repository)\n-\t\treturn 0;\n-\treturn oid_object_info_extended(r, oid, NULL, flags) >= 0;\n-}\n-\n-int repo_has_object_file(struct repository *r,\n-\t\t\t const struct object_id *oid)\n-{\n-\treturn repo_has_object_file_with_flags(r, oid, 0);\n-}\n-\n void assert_oid_type(const struct object_id *oid, enum object_type expect)\n {\n \tenum object_type type = oid_object_info(the_repository, oid, NULL);\ndiff --git a/object-store.h b/object-store.h\nindex c6055376f49..2330374990b 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -280,23 +280,6 @@ enum {\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags);\n \n-/*\n- * These macros and functions are deprecated. If checking existence for an\n- * object that is likely to be missing and/or whose absence is relatively\n- * inconsequential (or is consequential but the caller is prepared to handle\n- * it), use has_object(), which has better defaults (no lazy fetch in a partial\n- * clone and no rechecking of packed storage). In the unlikely event that a\n- * caller needs to assert existence of an object that it fully expects to\n- * exist, and wants to trigger a lazy fetch in a partial clone, use\n- * oid_object_info_extended() with a NULL struct object_info.\n- *\n- * These functions can be removed once all callers have migrated to\n- * has_object() and/or oid_object_info_extended().\n- */\n-int repo_has_object_file(struct repository *r, const struct object_id *oid);\n-int repo_has_object_file_with_flags(struct repository *r,\n-\t\t\t\t const struct object_id *oid, int flags);\n-\n void assert_oid_type(const struct object_id *oid, enum object_type expect);\n \n /*\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516585","messageId":"CAOLa=ZS8n+MTG1e=zViEL944yuO_UuqnmT9cTqmTkktHoUiirA@mail.gmail.com","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-3-81f8411a5d08@pks.im","subject":"Re: [PATCH 03/13] object-store: move and rename `odb_pack_keep()`","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-23T10:03:09Z","receivedAt":"2025-04-23T10:03:11Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> The function `odb_pack_keep()` creates a file at the passed-in path. If\n> this fails, then the function re-tries by first creating any potentially\n> missing leading directoriesk and then trying to create the file once\n\ns/directoriesk/directories\n\n> again. As such, this function doesn't host any kind of logic that is\n> specific to the object store, but is rather a generic helper function.\n>\n> Rename the function to `safe_create_file_with_leading_directories()` and\n> move it into \"path.c\". While at it, refactor it so that it loses its\n> dependency on `the_repository`.\n>\n\nRest of the patch looks good.\n"},{"id":"516592","messageId":"CAOLa=ZRv5WXHQYRg_19ED94+9QVSzi_EP_DAOXNPE4+4sB4P-A@mail.gmail.com","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-5-81f8411a5d08@pks.im","subject":"Re: [PATCH 05/13] object-store: allow fetching objects via `has_object()`","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-23T10:07:46Z","receivedAt":"2025-04-23T10:07:49Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> We're about to fully remove `repo_has_object_file()` in favor of\n> `has_object()` given that the latter has better defaults: it neither\n> reloads packfiles by default nor does it fetch any promised objects in\n> case they are missing.\n>\n> The latter usecase keeps us from converting a couple of callsites that\n> currently do fetch objects though. It is not really clear whether _all_\n> of those callsites should be fetching objects, but for a subset of them\n> it is the desired behaviour indeed.\n>\n> Introduce a new flag `HAS_OBJECT_FETCH_PROMISOR` that causes the\n> function to optionally fetch missing objects which are part of a\n> promisor pack.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> object-store.c | 9 ++++++---\n> object-store.h | 10 +++++++---\n> 2 files changed, 13 insertions(+), 6 deletions(-)\n>\n> diff --git a/object-store.c b/object-store.c\n> index 0cbad5a19a0..0d873868a6d 100644\n> --- a/object-store.c\n> +++ b/object-store.c\n> @@ -937,12 +937,15 @@ void *read_object_with_reference(struct repository *r,\n> int has_object(struct repository *r, const struct object_id *oid,\n> \t unsigned flags)\n> {\n> -\tint quick = !(flags & HAS_OBJECT_RECHECK_PACKED);\n> -\tunsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |\n> -\t\t(quick ? OBJECT_INFO_QUICK : 0);\n> +\tunsigned object_info_flags = 0;\n>\n> \tif (!startup_info->have_repository)\n> \t\treturn 0;\n> +\tif (!(flags & HAS_OBJECT_RECHECK_PACKED))\n> +\t\tobject_info_flags |= OBJECT_INFO_QUICK;\n> +\tif (!(flags & HAS_OBJECT_FETCH_PROMISOR))\n> +\t\tobject_info_flags |= OBJECT_INFO_SKIP_FETCH_OBJECT;\n> +\n\nThis already is much easier to read.\n\n> \treturn oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;\n> }\n>\n> diff --git a/object-store.h b/object-store.h\n> index 5bbdaba92d1..c6055376f49 100644\n> --- a/object-store.h\n> +++ b/object-store.h\n> @@ -266,12 +266,16 @@ int oid_object_info_extended(struct repository *r,\n> \t\t\t const struct object_id *,\n> \t\t\t struct object_info *, unsigned flags);\n>\n> -/* Retry packed storage after checking packed and loose storage */\n> -#define HAS_OBJECT_RECHECK_PACKED 1\n> +enum {\n> +\t/* Retry packed storage after checking packed and loose storage */\n> +\tHAS_OBJECT_RECHECK_PACKED = (1 << 0),\n> +\t/* Allow fetching the object in case the repository has a promisor remote. */\n> +\tHAS_OBJECT_FETCH_PROMISOR = (1 << 1),\n> +};\n>\n> /*\n> * Returns 1 if the object exists. This function will not lazily fetch objects\n> - * in a partial clone.\n> + * in a partial clone by default.\n> */\n> int has_object(struct repository *r, const struct object_id *oid,\n> \t unsigned flags);\n>\n> --\n> 2.49.0.901.g37484f566f.dirty\n\nOkay so we add the functionality here, and possibly use it in the future\npatches. Good.\n"},{"id":"516613","messageId":"CAOLa=ZQh+agLJYEGWU2O2=iRnJy94y9Cud3XRVaGVHC4YdbBZw@mail.gmail.com","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-7-81f8411a5d08@pks.im","subject":"Re: [PATCH 07/13] builtin/index-pack: don't fetch promised objects for collision check","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-23T17:08:05Z","receivedAt":"2025-04-23T17:08:07Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> Any packed objects indexed via git-index-pack(1) are subject to a\n> collision check. This collision check has the intent to determine\n> whether we already have an object with the same object ID, but different\n> contents in the repository.\n>\n> The check whether the collision check is really needed is performed via\n> `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)`. \\\n>\n\nNit: this was a little confusing at first, until I saw the code. So what\nthis means is that the collision check is only performed, iff\n`repo_has_object_file_with_flags(...)` returns true.\n\nI think the confusing part was 'is performed via', perhaps:\n\n The collision check is only performed, if\n repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK) returns a\n truthy value.\n\nBut it is okay as is too!\n\n> But unless\n> explicitly told otherwise via `OBJECT_INFO_SKIP_FETCH_OBJECT`, this\n> function will also cause us to fetch the object ID in case it is part of\n> a promisor pack. As such, we may end up fetching the object only to\n> check whether the fetched object and the object that we're indexing have\n> the same content.\n>\n\nSo us fetching the object is pointless, since we only care about the\n'does it exist' part and not really what it contains. In that case,\nshouldn't this be s/same content/same oid/?\n\n> This behaviour is highly dubious and more likely than not unintended.\n> Fix it by converting to `has_object()`, which knows to neither reload\n> packfiles nor to fetch promisor objects by default.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> builtin/index-pack.c | 4 +---\n> 1 file changed, 1 insertion(+), 3 deletions(-)\n>\n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index f49431d626b..805b7aa1e28 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -892,9 +892,7 @@ static void sha1_object(const void *data, struct object_entry *obj_entry,\n>\n> \tif (startup_info->have_repository) {\n> \t\tread_lock();\n> -\t\tcollision_test_needed =\n> -\t\t\trepo_has_object_file_with_flags(the_repository, oid,\n> -\t\t\t\t\t\t\tOBJECT_INFO_QUICK);\n> +\t\tcollision_test_needed = has_object(the_repository, oid, 0);\n> \t\tread_unlock();\n> \t}\n>\n>\n> --\n> 2.49.0.901.g37484f566f.dirty\n"},{"id":"516614","messageId":"CAOLa=ZRn=+kSzHqME4AM-O-Z_05bojkHu5_hymNpH1aoLb2zEQ@mail.gmail.com","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-9-81f8411a5d08@pks.im","subject":"Re: [PATCH 09/13] refs: don't fetch promisor objects in `ref_resolves_to_object()`","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-23T17:11:57Z","receivedAt":"2025-04-23T17:11:59Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> Similar to the preceding commit, don't try to fetch objects pointed to\n> by references. Any reference whose object does not exist is broken by\n> definition an, so we should report it accordingly.\n>\n\ns/an//\n\n> Signed-off-by: Patrick Steinhardt \n> ---\n> refs.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/refs.c b/refs.c\n> index 6559db37890..0492cf0d13a 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -376,7 +376,7 @@ int ref_resolves_to_object(const char *refname,\n> {\n> \tif (flags & REF_ISBROKEN)\n> \t\treturn 0;\n> -\tif (!repo_has_object_file(repo, oid)) {\n> +\tif (!has_object(repo, oid, HAS_OBJECT_RECHECK_PACKED)) {\n> \t\terror(_(\"%s does not point to a valid object!\"), refname);\n> \t\treturn 0;\n> \t}\n>\n> --\n> 2.49.0.901.g37484f566f.dirty\n"},{"id":"516615","messageId":"CAOLa=ZQ45v33yj2bUNrN6ZJYawR2KVyjHAtsY0RZ58fkhkTbSw@mail.gmail.com","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"Re: [PATCH 00/13] object-store: a handful of cleanups","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-23T17:20:18Z","receivedAt":"2025-04-23T17:20:19Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> Hi,\n>\n> this patch series contains a handful of cleanups to the object store\n> subsystem:\n>\n> - A couple of definitions are moved out of \"object-store.h\" as they\n> belong to other subsystems.\n>\n> - Some functions are dropped and/or renamed.\n>\n> - The biggest part is the removal of `repo_has_object_file()`. This\n> function and its `_with_flags()` variant are marked as deprecated,\n> with the replacement being `has_object()`. The benefit of that\n> function is that it doesn't reload packfiles and doesn't fetch\n> promisor objects by default so that it becomes more explicit when\n> one really wants to do so.\n>\n> These cleanups are in preparation for getting rid of `the_repository` in\n> \"object-store.c\".\n>\n\nApart from the few nits I mentioned, the series looks great! I must say\nthe split of commits was really nice to go through :)\n\n[snip]\n"},{"id":"516736","messageId":"aAs0F58Hlpn6WI6b@pks.im","threadId":"63335","inReplyTo":"CAOLa=ZQh+agLJYEGWU2O2=iRnJy94y9Cud3XRVaGVHC4YdbBZw@mail.gmail.com","subject":"Re: [PATCH 07/13] builtin/index-pack: don't fetch promised objects for collision check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:04:55Z","receivedAt":"2025-04-25T07:05:04Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Apr 23, 2025 at 10:08:05AM -0700, Karthik Nayak wrote:\n> Patrick Steinhardt writes:\n> \n> > Any packed objects indexed via git-index-pack(1) are subject to a\n> > collision check. This collision check has the intent to determine\n> > whether we already have an object with the same object ID, but different\n> > contents in the repository.\n> >\n> > The check whether the collision check is really needed is performed via\n> > `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)`. \\\n> >\n> \n> Nit: this was a little confusing at first, until I saw the code. So what\n> this means is that the collision check is only performed, iff\n> `repo_has_object_file_with_flags(...)` returns true.\n> \n> I think the confusing part was 'is performed via', perhaps:\n> \n> The collision check is only performed, if\n> repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK) returns a\n> truthy value.\n> \n> But it is okay as is too!\n\nWill rephrase.\n\n> > But unless\n> > explicitly told otherwise via `OBJECT_INFO_SKIP_FETCH_OBJECT`, this\n> > function will also cause us to fetch the object ID in case it is part of\n> > a promisor pack. As such, we may end up fetching the object only to\n> > check whether the fetched object and the object that we're indexing have\n> > the same content.\n> >\n> \n> So us fetching the object is pointless, since we only care about the\n> 'does it exist' part and not really what it contains. In that case,\n> shouldn't this be s/same content/same oid/?\n\nNo, it really checks for the same content. It basically verifies that\nany pair of objects that:\n\n - Exist in the packfile that we're currently indexing.\n - And preexists in the local repository.\n\nActually have the same content.\n\nThe weird part is that we also do this for objects that don't yet exist\nin the repository, but which are promised to us. This causes us to fetch\nthem first only to verify that the fetched promised object has the same\ncontent as the packfile. And given that git-index-pack(1) would usually\nrun after a fetch, we end up verifying that the fetched object obtained\nfrom the promisor is the same as the fetched object obtained from the\npackfile. Which ultimately seems rather dubious to me.\n\nPatrick\n"},{"id":"516737","messageId":"aAs0lFpGqL6_a-y2@pks.im","threadId":"63335","inReplyTo":"CAOLa=ZQ45v33yj2bUNrN6ZJYawR2KVyjHAtsY0RZ58fkhkTbSw@mail.gmail.com","subject":"Re: [PATCH 00/13] object-store: a handful of cleanups","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:07:00Z","receivedAt":"2025-04-25T07:07:04Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Apr 23, 2025 at 10:20:18AM -0700, Karthik Nayak wrote:\n> Patrick Steinhardt writes:\n> \n> > Hi,\n> >\n> > this patch series contains a handful of cleanups to the object store\n> > subsystem:\n> >\n> > - A couple of definitions are moved out of \"object-store.h\" as they\n> > belong to other subsystems.\n> >\n> > - Some functions are dropped and/or renamed.\n> >\n> > - The biggest part is the removal of `repo_has_object_file()`. This\n> > function and its `_with_flags()` variant are marked as deprecated,\n> > with the replacement being `has_object()`. The benefit of that\n> > function is that it doesn't reload packfiles and doesn't fetch\n> > promisor objects by default so that it becomes more explicit when\n> > one really wants to do so.\n> >\n> > These cleanups are in preparation for getting rid of `the_repository` in\n> > \"object-store.c\".\n> >\n> \n> Apart from the few nits I mentioned, the series looks great! I must say\n> the split of commits was really nice to go through :)\n\nThanks for your review!\n\nPatrick\n"},{"id":"516738","messageId":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH v2 00/13] object-store: a handful of cleanups","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:51Z","receivedAt":"2025-04-25T07:08:59Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series contains a handful of cleanups to the object store\nsubsystem:\n\n - A couple of definitions are moved out of \"object-store.h\" as they\n belong to other subsystems.\n\n - Some functions are dropped and/or renamed.\n\n - The biggest part is the removal of `repo_has_object_file()`. This\n function and its `_with_flags()` variant are marked as deprecated,\n with the replacement being `has_object()`. The benefit of that\n function is that it doesn't reload packfiles and doesn't fetch\n promisor objects by default so that it becomes more explicit when\n one really wants to do so.\n\nThese cleanups are in preparation for getting rid of `the_repository` in\n\"object-store.c\".\n\nThe patch series is built on top of 4bbb303af69 (The seventh batch,\n2025-04-17) with ps/object-file-cleanup at 68cd492a3e6 (object-store:\nmerge \"object-store-ll.h\" and \"object-store.h\", 2025-04-15) merged into\nit.\n\nChanges in v2:\n - A handful of improvements for commit messages.\n - Link to v1: https://lore.kernel.org/r/20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (13):\n object-store: move `struct packed_git` into \"packfile.h\"\n object-store: drop `loose_object_path()`\n object-store: move and rename `odb_pack_keep()`\n object-store: move function declarations to their respective subsystems\n object-store: allow fetching objects via `has_object()`\n treewide: trivial conversions of `repo_has_object_file()`\n builtin/index-pack: don't fetch promised objects for collision check\n builtin/show-ref: don't fetch objects when printing refs\n refs: don't fetch promisor objects in `ref_resolves_to_object()`\n http-walker: don't fetch objects via promisor remotes\n list-objects: clarify how promised blobs are excluded\n bulk-checkin: don't fetch promised objects on write\n object-store: drop `repo_has_object_file()`\n\n builtin/cat-file.c | 3 +-\n builtin/clone.c | 4 +-\n builtin/count-objects.c | 2 +-\n builtin/fast-import.c | 3 +-\n builtin/fetch.c | 15 ++--\n builtin/gc.c | 2 +-\n builtin/index-pack.c | 6 +-\n builtin/receive-pack.c | 4 +-\n builtin/remote.c | 3 +-\n builtin/show-ref.c | 2 +-\n builtin/unpack-objects.c | 3 +-\n bulk-checkin.c | 2 +-\n cache-tree.c | 13 +++-\n fetch-pack.c | 7 +-\n http-push.c | 11 ++-\n http-walker.c | 7 +-\n http.c | 4 +-\n list-objects.c | 3 +-\n notes.c | 3 +-\n object-file.c | 4 +-\n object-file.h | 77 +++++++++++++++++++\n object-name.c | 2 +-\n object-store.c | 44 ++---------\n object-store.h | 191 +++--------------------------------------------\n pack-objects.h | 1 +\n packfile.h | 78 ++++++++++++++++++-\n path.c | 14 ++++\n path.h | 7 ++\n prune-packed.c | 2 +-\n reachable.c | 2 +-\n reflog.c | 3 +-\n refs.c | 2 +-\n remote.c | 2 +-\n send-pack.c | 5 +-\n shallow.c | 9 ++-\n upload-pack.c | 3 +-\n walker.c | 3 +-\n 37 files changed, 265 insertions(+), 281 deletions(-)\n\nRange-diff versus v1:\n\n 1: b4f8a00f4c4 = 1: 019c27227dc object-store: move `struct packed_git` into \"packfile.h\"\n 2: 8684c481949 = 2: b372e8214de object-store: drop `loose_object_path()`\n 3: f4f5127f44f ! 3: fa51af2ee24 object-store: move and rename `odb_pack_keep()`\n @@ Commit message\n \n The function `odb_pack_keep()` creates a file at the passed-in path. If\n this fails, then the function re-tries by first creating any potentially\n - missing leading directoriesk and then trying to create the file once\n + missing leading directories and then trying to create the file once\n again. As such, this function doesn't host any kind of logic that is\n specific to the object store, but is rather a generic helper function.\n \n 4: 9f94d3c4780 = 4: 04ad9a1b228 object-store: move function declarations to their respective subsystems\n 5: 0a187fe90db = 5: 3d45b334f4b object-store: allow fetching objects via `has_object()`\n 6: 4ebdf7510d2 = 6: 6101dfc393a treewide: trivial conversions of `repo_has_object_file()`\n 7: 739de6f8c67 ! 7: 35eca639ba4 builtin/index-pack: don't fetch promised objects for collision check\n @@ Commit message\n whether we already have an object with the same object ID, but different\n contents in the repository.\n \n - The check whether the collision check is really needed is performed via\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)`. But unless\n - explicitly told otherwise via `OBJECT_INFO_SKIP_FETCH_OBJECT`, this\n - function will also cause us to fetch the object ID in case it is part of\n - a promisor pack. As such, we may end up fetching the object only to\n - check whether the fetched object and the object that we're indexing have\n - the same content.\n + The check whether the collision check is really needed is only performed\n + in case `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)` tells\n + us that the object exists. But unless explicitly told otherwise by\n + passing `OBJECT_INFO_SKIP_FETCH_OBJECT`, this function will also cause\n + us to fetch the object in case it is part of a promisor pack. As such,\n + we may end up fetching the object only to check whether the fetched\n + object and the object that we're indexing have the same content.\n \n This behaviour is highly dubious and more likely than not unintended.\n Fix it by converting to `has_object()`, which knows to neither reload\n 8: 0a79bfdbf14 = 8: 9975d86c59d builtin/show-ref: don't fetch objects when printing refs\n 9: 3a1ebffcdb0 ! 9: 41ad3e7ede2 refs: don't fetch promisor objects in `ref_resolves_to_object()`\n @@ Commit message\n \n Similar to the preceding commit, don't try to fetch objects pointed to\n by references. Any reference whose object does not exist is broken by\n - definition an, so we should report it accordingly.\n + definition, so we should report it accordingly.\n \n Signed-off-by: Patrick Steinhardt \n \n10: a21cfe3dc91 = 10: e7bb54ba5bd http-walker: don't fetch objects via promisor remotes\n11: b939734e9e6 = 11: bef052ff785 list-objects: clarify how promised blobs are excluded\n12: 8e8a5041af3 = 12: 45023c6e96f bulk-checkin: don't fetch promised objects on write\n13: 68deca60383 = 13: c263ae0f4cf object-store: drop `repo_has_object_file()`\n\n---\nbase-commit: ca819c0751cedd1713334882e4c83687f8478a54\nchange-id: 20250422-pks-object-store-cleanups-5a6077014155\n\n"},{"id":"516739","messageId":"20250425-pks-object-store-cleanups-v2-1-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 01/13] object-store: move `struct packed_git` into \"packfile.h\"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:52Z","receivedAt":"2025-04-25T07:08:59Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The \"object-store.h\" header contains the definition of `struct\npacked_git`. As this structure hosts all kind of information about a\nspecific packfile it is arguably a bit out of place in a generic place\nlike \"object-store.h\".\n\nMove the structure as well as `pack_map_entry_cmp()` into \"packfile.h\".\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.h | 59 +---------------------------------------------------------\n pack-objects.h | 1 +\n packfile.h | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-\n 3 files changed, 60 insertions(+), 59 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 46961dc9542..e04469a85fb 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -92,65 +92,8 @@ struct oidtree *odb_loose_cache(struct object_directory *odb,\n /* Empty the loose object cache for the specified object directory. */\n void odb_clear_loose_cache(struct object_directory *odb);\n \n-struct packed_git {\n-\tstruct hashmap_entry packmap_ent;\n-\tstruct packed_git *next;\n-\tstruct list_head mru;\n-\tstruct pack_window *windows;\n-\toff_t pack_size;\n-\tconst void *index_data;\n-\tsize_t index_size;\n-\tuint32_t num_objects;\n-\tsize_t crc_offset;\n-\tstruct oidset bad_objects;\n-\tint index_version;\n-\ttime_t mtime;\n-\tint pack_fd;\n-\tint index; /* for builtin/pack-objects.c */\n-\tunsigned pack_local:1,\n-\t\t pack_keep:1,\n-\t\t pack_keep_in_core:1,\n-\t\t freshened:1,\n-\t\t do_not_close:1,\n-\t\t pack_promisor:1,\n-\t\t multi_pack_index:1,\n-\t\t is_cruft:1;\n-\tunsigned char hash[GIT_MAX_RAWSZ];\n-\tstruct revindex_entry *revindex;\n-\tconst uint32_t *revindex_data;\n-\tconst uint32_t *revindex_map;\n-\tsize_t revindex_size;\n-\t/*\n-\t * mtimes_map points at the beginning of the memory mapped region of\n-\t * this pack's corresponding .mtimes file, and mtimes_size is the size\n-\t * of that .mtimes file\n-\t */\n-\tconst uint32_t *mtimes_map;\n-\tsize_t mtimes_size;\n-\n-\t/* repo denotes the repository this packfile belongs to */\n-\tstruct repository *repo;\n-\n-\t/* something like \".git/objects/pack/xxxxx.pack\" */\n-\tchar pack_name[FLEX_ARRAY]; /* more */\n-};\n-\n+struct packed_git;\n struct multi_pack_index;\n-\n-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,\n-\t\t\t\t const struct hashmap_entry *entry,\n-\t\t\t\t const struct hashmap_entry *entry2,\n-\t\t\t\t const void *keydata)\n-{\n-\tconst char *key = keydata;\n-\tconst struct packed_git *pg1, *pg2;\n-\n-\tpg1 = container_of(entry, const struct packed_git, packmap_ent);\n-\tpg2 = container_of(entry2, const struct packed_git, packmap_ent);\n-\n-\treturn strcmp(pg1->pack_name, key ? key : pg2->pack_name);\n-}\n-\n struct cached_object_entry;\n \n struct raw_object_store {\ndiff --git a/pack-objects.h b/pack-objects.h\nindex d1c4ae7f9b6..475a2d67ce3 100644\n--- a/pack-objects.h\n+++ b/pack-objects.h\n@@ -4,6 +4,7 @@\n #include \"object-store.h\"\n #include \"thread-utils.h\"\n #include \"pack.h\"\n+#include \"packfile.h\"\n \n struct repository;\n \ndiff --git a/packfile.h b/packfile.h\nindex 25097213d06..05499382397 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -1,13 +1,70 @@\n #ifndef PACKFILE_H\n #define PACKFILE_H\n \n+#include \"list.h\"\n #include \"object.h\"\n #include \"oidset.h\"\n \n /* in object-store.h */\n-struct packed_git;\n struct object_info;\n \n+struct packed_git {\n+\tstruct hashmap_entry packmap_ent;\n+\tstruct packed_git *next;\n+\tstruct list_head mru;\n+\tstruct pack_window *windows;\n+\toff_t pack_size;\n+\tconst void *index_data;\n+\tsize_t index_size;\n+\tuint32_t num_objects;\n+\tsize_t crc_offset;\n+\tstruct oidset bad_objects;\n+\tint index_version;\n+\ttime_t mtime;\n+\tint pack_fd;\n+\tint index; /* for builtin/pack-objects.c */\n+\tunsigned pack_local:1,\n+\t\t pack_keep:1,\n+\t\t pack_keep_in_core:1,\n+\t\t freshened:1,\n+\t\t do_not_close:1,\n+\t\t pack_promisor:1,\n+\t\t multi_pack_index:1,\n+\t\t is_cruft:1;\n+\tunsigned char hash[GIT_MAX_RAWSZ];\n+\tstruct revindex_entry *revindex;\n+\tconst uint32_t *revindex_data;\n+\tconst uint32_t *revindex_map;\n+\tsize_t revindex_size;\n+\t/*\n+\t * mtimes_map points at the beginning of the memory mapped region of\n+\t * this pack's corresponding .mtimes file, and mtimes_size is the size\n+\t * of that .mtimes file\n+\t */\n+\tconst uint32_t *mtimes_map;\n+\tsize_t mtimes_size;\n+\n+\t/* repo denotes the repository this packfile belongs to */\n+\tstruct repository *repo;\n+\n+\t/* something like \".git/objects/pack/xxxxx.pack\" */\n+\tchar pack_name[FLEX_ARRAY]; /* more */\n+};\n+\n+static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,\n+\t\t\t\t const struct hashmap_entry *entry,\n+\t\t\t\t const struct hashmap_entry *entry2,\n+\t\t\t\t const void *keydata)\n+{\n+\tconst char *key = keydata;\n+\tconst struct packed_git *pg1, *pg2;\n+\n+\tpg1 = container_of(entry, const struct packed_git, packmap_ent);\n+\tpg2 = container_of(entry2, const struct packed_git, packmap_ent);\n+\n+\treturn strcmp(pg1->pack_name, key ? key : pg2->pack_name);\n+}\n+\n struct pack_window {\n \tstruct pack_window *next;\n \tunsigned char *base;\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516740","messageId":"20250425-pks-object-store-cleanups-v2-2-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 02/13] object-store: drop `loose_object_path()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:53Z","receivedAt":"2025-04-25T07:09:00Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The function `loose_object_path()` is a trivial wrapper around\n`odb_loose_path()`, with the only exception that it always uses the\nprimary object database of the given repository. This doesn't really add\na ton of value though, so let's drop the function and inline it at every\ncallsite.\n\nSigned-off-by: Patrick Steinhardt \n---\n http-walker.c | 3 ++-\n http.c | 4 ++--\n object-file.c | 4 ++--\n object-file.h | 4 ++++\n object-store.c | 6 ------\n object-store.h | 7 -------\n 6 files changed, 10 insertions(+), 18 deletions(-)\n\ndiff --git a/http-walker.c b/http-walker.c\nindex 882cae19c24..95458e2f638 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -9,6 +9,7 @@\n #include \"list.h\"\n #include \"transport.h\"\n #include \"packfile.h\"\n+#include \"object-file.h\"\n #include \"object-store.h\"\n \n struct alt_base {\n@@ -540,7 +541,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)\n \t\tret = error(\"File %s has bad hash\", hex);\n \t} else if (req->rename < 0) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tloose_object_path(the_repository, &buf, &req->oid);\n+\t\todb_loose_path(the_repository->objects->odb, &buf, &req->oid);\n \t\tret = error(\"unable to write sha1 filename %s\", buf.buf);\n \t\tstrbuf_release(&buf);\n \t}\ndiff --git a/http.c b/http.c\nindex 0c411380425..3c029cf8947 100644\n--- a/http.c\n+++ b/http.c\n@@ -2662,7 +2662,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \toidcpy(&freq->oid, oid);\n \tfreq->localfile = -1;\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \tstrbuf_addf(&freq->tmpfile, \"%s.temp\", filename.buf);\n \n \tstrbuf_addf(&prevfile, \"%s.prev\", filename.buf);\n@@ -2814,7 +2814,7 @@ int finish_http_object_request(struct http_object_request *freq)\n \t\tunlink_or_warn(freq->tmpfile.buf);\n \t\treturn -1;\n \t}\n-\tloose_object_path(the_repository, &filename, &freq->oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, &freq->oid);\n \tfreq->rename = finalize_object_file(freq->tmpfile.buf, filename.buf);\n \tstrbuf_release(&filename);\n \ndiff --git a/object-file.c b/object-file.c\nindex 9cc3a24a40d..dc56a4766df 100644\n--- a/object-file.c\n+++ b/object-file.c\n@@ -932,7 +932,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr,\n \tif (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT))\n \t\tprepare_loose_object_bulk_checkin();\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \n \tfd = start_loose_object_common(&tmp_file, filename.buf, flags,\n \t\t\t\t &stream, compressed, sizeof(compressed),\n@@ -1079,7 +1079,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len,\n \t\tgoto cleanup;\n \t}\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \n \t/* We finally know the object path, and create the missing dir. */\n \tdirlen = directory_size(filename.buf);\ndiff --git a/object-file.h b/object-file.h\nindex c002fbe2345..0a7b6b9f9d9 100644\n--- a/object-file.h\n+++ b/object-file.h\n@@ -25,6 +25,10 @@ int index_path(struct index_state *istate, struct object_id *oid, const char *pa\n \n struct object_directory;\n \n+/*\n+ * Put in `buf` the name of the file in the local object database that\n+ * would be used to store a loose object with the specified oid.\n+ */\n const char *odb_loose_path(struct object_directory *odb,\n \t\t\t struct strbuf *buf,\n \t\t\t const struct object_id *oid);\ndiff --git a/object-store.c b/object-store.c\nindex 6ab50d25d3e..e5cfb8c0079 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -96,12 +96,6 @@ int odb_pack_keep(const char *name)\n \treturn open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n }\n \n-const char *loose_object_path(struct repository *r, struct strbuf *buf,\n-\t\t\t const struct object_id *oid)\n-{\n-\treturn odb_loose_path(r->objects->odb, buf, oid);\n-}\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\ndiff --git a/object-store.h b/object-store.h\nindex e04469a85fb..5668de62d01 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -196,13 +196,6 @@ int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n */\n int odb_pack_keep(const char *name);\n \n-/*\n- * Put in `buf` the name of the file in the local object database that\n- * would be used to store a loose object with the specified oid.\n- */\n-const char *loose_object_path(struct repository *r, struct strbuf *buf,\n-\t\t\t const struct object_id *oid);\n-\n void *map_loose_object(struct repository *r, const struct object_id *oid,\n \t\t unsigned long *size);\n \n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516741","messageId":"20250425-pks-object-store-cleanups-v2-3-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 03/13] object-store: move and rename `odb_pack_keep()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:54Z","receivedAt":"2025-04-25T07:09:01Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The function `odb_pack_keep()` creates a file at the passed-in path. If\nthis fails, then the function re-tries by first creating any potentially\nmissing leading directories and then trying to create the file once\nagain. As such, this function doesn't host any kind of logic that is\nspecific to the object store, but is rather a generic helper function.\n\nRename the function to `safe_create_file_with_leading_directories()` and\nmove it into \"path.c\". While at it, refactor it so that it loses its\ndependency on `the_repository`.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/fast-import.c | 3 ++-\n builtin/index-pack.c | 2 +-\n object-store.c | 13 -------------\n object-store.h | 7 -------\n path.c | 14 ++++++++++++++\n path.h | 7 +++++++\n 6 files changed, 24 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin/fast-import.c b/builtin/fast-import.c\nindex c1e198f4e34..b2839c5f439 100644\n--- a/builtin/fast-import.c\n+++ b/builtin/fast-import.c\n@@ -811,7 +811,8 @@ static char *keep_pack(const char *curr_index_name)\n \tint keep_fd;\n \n \todb_pack_name(pack_data->repo, &name, pack_data->hash, \"keep\");\n-\tkeep_fd = odb_pack_keep(name.buf);\n+\tkeep_fd = safe_create_file_with_leading_directories(pack_data->repo,\n+\t\t\t\t\t\t\t name.buf);\n \tif (keep_fd < 0)\n \t\tdie_errno(\"cannot create keep file\");\n \twrite_or_die(keep_fd, keep_msg, strlen(keep_msg));\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 60a8ee05dbc..f49431d626b 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1565,7 +1565,7 @@ static void write_special_file(const char *suffix, const char *msg,\n \telse\n \t\tfilename = odb_pack_name(the_repository, &name_buf, hash, suffix);\n \n-\tfd = odb_pack_keep(filename);\n+\tfd = safe_create_file_with_leading_directories(the_repository, filename);\n \tif (fd < 0) {\n \t\tif (errno != EEXIST)\n \t\t\tdie_errno(_(\"cannot write %s file '%s'\"),\ndiff --git a/object-store.c b/object-store.c\nindex e5cfb8c0079..0cbad5a19a0 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -83,19 +83,6 @@ int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)\n \treturn xmkstemp_mode(temp_filename->buf, mode);\n }\n \n-int odb_pack_keep(const char *name)\n-{\n-\tint fd;\n-\n-\tfd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n-\tif (0 <= fd)\n-\t\treturn fd;\n-\n-\t/* slow path */\n-\tsafe_create_leading_directories_const(the_repository, name);\n-\treturn open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n-}\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\ndiff --git a/object-store.h b/object-store.h\nindex 5668de62d01..aa8fc63043e 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -189,13 +189,6 @@ void raw_object_store_clear(struct raw_object_store *o);\n */\n int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n \n-/*\n- * Create a pack .keep file named \"name\" (which should generally be the output\n- * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on\n- * error.\n- */\n-int odb_pack_keep(const char *name);\n-\n void *map_loose_object(struct repository *r, const struct object_id *oid,\n \t\t unsigned long *size);\n \ndiff --git a/path.c b/path.c\nindex 4505bb78e8b..3b598b2847f 100644\n--- a/path.c\n+++ b/path.c\n@@ -1011,6 +1011,20 @@ enum scld_error safe_create_leading_directories_const(struct repository *repo,\n \treturn result;\n }\n \n+int safe_create_file_with_leading_directories(struct repository *repo,\n+\t\t\t\t\t const char *path)\n+{\n+\tint fd;\n+\n+\tfd = open(path, O_RDWR|O_CREAT|O_EXCL, 0600);\n+\tif (0 <= fd)\n+\t\treturn fd;\n+\n+\t/* slow path */\n+\tsafe_create_leading_directories_const(repo, path);\n+\treturn open(path, O_RDWR|O_CREAT|O_EXCL, 0600);\n+}\n+\n static int have_same_root(const char *path1, const char *path2)\n {\n \tint is_abs1, is_abs2;\ndiff --git a/path.h b/path.h\nindex fd1a194b060..e67348f2539 100644\n--- a/path.h\n+++ b/path.h\n@@ -266,6 +266,13 @@ enum scld_error safe_create_leading_directories_const(struct repository *repo,\n \t\t\t\t\t\t const char *path);\n enum scld_error safe_create_leading_directories_no_share(char *path);\n \n+/*\n+ * Create a file, potentially creating its leading directories in case they\n+ * don't exist. Returns the return value of the open(3p) call.\n+ */\n+int safe_create_file_with_leading_directories(struct repository *repo,\n+\t\t\t\t\t const char *path);\n+\n # ifdef USE_THE_REPOSITORY_VARIABLE\n # include \"strbuf.h\"\n # include \"repository.h\"\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516742","messageId":"20250425-pks-object-store-cleanups-v2-4-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 04/13] object-store: move function declarations to their respective subsystems","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:55Z","receivedAt":"2025-04-25T07:09:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"We carry declarations for a couple of functions in \"object-store.h\" that\nare not defined in \"object-store.c\", but in a different subsystem. Move\nthese declarations to the respective headers whose matching code files\ncarry the corresponding definition.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/count-objects.c | 2 +-\n builtin/gc.c | 2 +-\n object-file.h | 73 +++++++++++++++++++++++++++++++++++++++\n object-name.c | 2 +-\n object-store.h | 91 +------------------------------------------------\n packfile.h | 19 +++++++++++\n prune-packed.c | 2 +-\n reachable.c | 2 +-\n 8 files changed, 98 insertions(+), 95 deletions(-)\n\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex 0bb5360b2f2..a88c0c9c09a 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -12,7 +12,7 @@\n #include \"parse-options.h\"\n #include \"quote.h\"\n #include \"packfile.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n \n static unsigned long garbage;\n static off_t size_garbage;\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex b5ce1d32766..4d428f3253d 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -28,7 +28,7 @@\n #include \"commit.h\"\n #include \"commit-graph.h\"\n #include \"packfile.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"pack.h\"\n #include \"pack-objects.h\"\n #include \"path.h\"\ndiff --git a/object-file.h b/object-file.h\nindex 0a7b6b9f9d9..de6dd205ed8 100644\n--- a/object-file.h\n+++ b/object-file.h\n@@ -3,6 +3,7 @@\n \n #include \"git-zlib.h\"\n #include \"object.h\"\n+#include \"object-store.h\"\n \n struct index_state;\n \n@@ -25,6 +26,16 @@ int index_path(struct index_state *istate, struct object_id *oid, const char *pa\n \n struct object_directory;\n \n+/*\n+ * Populate and return the loose object cache array corresponding to the\n+ * given object ID.\n+ */\n+struct oidtree *odb_loose_cache(struct object_directory *odb,\n+\t\t\t\tconst struct object_id *oid);\n+\n+/* Empty the loose object cache for the specified object directory. */\n+void odb_clear_loose_cache(struct object_directory *odb);\n+\n /*\n * Put in `buf` the name of the file in the local object database that\n * would be used to store a loose object with the specified oid.\n@@ -42,6 +53,68 @@ int has_loose_object_nonlocal(const struct object_id *);\n \n int has_loose_object(const struct object_id *);\n \n+void *map_loose_object(struct repository *r, const struct object_id *oid,\n+\t\t unsigned long *size);\n+\n+/*\n+ * Iterate over the files in the loose-object parts of the object\n+ * directory \"path\", triggering the following callbacks:\n+ *\n+ * - loose_object is called for each loose object we find.\n+ *\n+ * - loose_cruft is called for any files that do not appear to be\n+ * loose objects. Note that we only look in the loose object\n+ * directories \"objects/[0-9a-f]{2}/\", so we will not report\n+ * \"objects/foobar\" as cruft.\n+ *\n+ * - loose_subdir is called for each top-level hashed subdirectory\n+ * of the object directory (e.g., \"$OBJDIR/f0\"). It is called\n+ * after the objects in the directory are processed.\n+ *\n+ * Any callback that is NULL will be ignored. Callbacks returning non-zero\n+ * will end the iteration.\n+ *\n+ * In the \"buf\" variant, \"path\" is a strbuf which will also be used as a\n+ * scratch buffer, but restored to its original contents before\n+ * the function returns.\n+ */\n+typedef int each_loose_object_fn(const struct object_id *oid,\n+\t\t\t\t const char *path,\n+\t\t\t\t void *data);\n+typedef int each_loose_cruft_fn(const char *basename,\n+\t\t\t\tconst char *path,\n+\t\t\t\tvoid *data);\n+typedef int each_loose_subdir_fn(unsigned int nr,\n+\t\t\t\t const char *path,\n+\t\t\t\t void *data);\n+int for_each_file_in_obj_subdir(unsigned int subdir_nr,\n+\t\t\t\tstruct strbuf *path,\n+\t\t\t\teach_loose_object_fn obj_cb,\n+\t\t\t\teach_loose_cruft_fn cruft_cb,\n+\t\t\t\teach_loose_subdir_fn subdir_cb,\n+\t\t\t\tvoid *data);\n+int for_each_loose_file_in_objdir(const char *path,\n+\t\t\t\t each_loose_object_fn obj_cb,\n+\t\t\t\t each_loose_cruft_fn cruft_cb,\n+\t\t\t\t each_loose_subdir_fn subdir_cb,\n+\t\t\t\t void *data);\n+int for_each_loose_file_in_objdir_buf(struct strbuf *path,\n+\t\t\t\t each_loose_object_fn obj_cb,\n+\t\t\t\t each_loose_cruft_fn cruft_cb,\n+\t\t\t\t each_loose_subdir_fn subdir_cb,\n+\t\t\t\t void *data);\n+\n+/*\n+ * Iterate over all accessible loose objects without respect to\n+ * reachability. By default, this includes both local and alternate objects.\n+ * The order in which objects are visited is unspecified.\n+ *\n+ * Any flags specific to packs are ignored.\n+ */\n+int for_each_loose_object(each_loose_object_fn, void *,\n+\t\t\t enum for_each_object_flags flags);\n+\n+\n /**\n * format_object_header() is a thin wrapper around s xsnprintf() that\n * writes the initial \" \" part of the loose object\ndiff --git a/object-name.c b/object-name.c\nindex 2c751a5352a..9288b2dd245 100644\n--- a/object-name.c\n+++ b/object-name.c\n@@ -19,7 +19,7 @@\n #include \"oidtree.h\"\n #include \"packfile.h\"\n #include \"pretty.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"read-cache-ll.h\"\n #include \"repo-settings.h\"\n #include \"repository.h\"\ndiff --git a/object-store.h b/object-store.h\nindex aa8fc63043e..5bbdaba92d1 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -82,16 +82,6 @@ struct object_directory *set_temporary_primary_odb(const char *dir, int will_des\n */\n void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);\n \n-/*\n- * Populate and return the loose object cache array corresponding to the\n- * given object ID.\n- */\n-struct oidtree *odb_loose_cache(struct object_directory *odb,\n-\t\t\t\t const struct object_id *oid);\n-\n-/* Empty the loose object cache for the specified object directory. */\n-void odb_clear_loose_cache(struct object_directory *odb);\n-\n struct packed_git;\n struct multi_pack_index;\n struct cached_object_entry;\n@@ -189,9 +179,6 @@ void raw_object_store_clear(struct raw_object_store *o);\n */\n int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n \n-void *map_loose_object(struct repository *r, const struct object_id *oid,\n-\t\t unsigned long *size);\n-\n void *repo_read_object_file(struct repository *r,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type *type,\n@@ -340,56 +327,7 @@ static inline void obj_read_unlock(void)\n \tif(obj_read_use_lock)\n \t\tpthread_mutex_unlock(&obj_read_mutex);\n }\n-\n-/*\n- * Iterate over the files in the loose-object parts of the object\n- * directory \"path\", triggering the following callbacks:\n- *\n- * - loose_object is called for each loose object we find.\n- *\n- * - loose_cruft is called for any files that do not appear to be\n- * loose objects. Note that we only look in the loose object\n- * directories \"objects/[0-9a-f]{2}/\", so we will not report\n- * \"objects/foobar\" as cruft.\n- *\n- * - loose_subdir is called for each top-level hashed subdirectory\n- * of the object directory (e.g., \"$OBJDIR/f0\"). It is called\n- * after the objects in the directory are processed.\n- *\n- * Any callback that is NULL will be ignored. Callbacks returning non-zero\n- * will end the iteration.\n- *\n- * In the \"buf\" variant, \"path\" is a strbuf which will also be used as a\n- * scratch buffer, but restored to its original contents before\n- * the function returns.\n- */\n-typedef int each_loose_object_fn(const struct object_id *oid,\n-\t\t\t\t const char *path,\n-\t\t\t\t void *data);\n-typedef int each_loose_cruft_fn(const char *basename,\n-\t\t\t\tconst char *path,\n-\t\t\t\tvoid *data);\n-typedef int each_loose_subdir_fn(unsigned int nr,\n-\t\t\t\t const char *path,\n-\t\t\t\t void *data);\n-int for_each_file_in_obj_subdir(unsigned int subdir_nr,\n-\t\t\t\tstruct strbuf *path,\n-\t\t\t\teach_loose_object_fn obj_cb,\n-\t\t\t\teach_loose_cruft_fn cruft_cb,\n-\t\t\t\teach_loose_subdir_fn subdir_cb,\n-\t\t\t\tvoid *data);\n-int for_each_loose_file_in_objdir(const char *path,\n-\t\t\t\t each_loose_object_fn obj_cb,\n-\t\t\t\t each_loose_cruft_fn cruft_cb,\n-\t\t\t\t each_loose_subdir_fn subdir_cb,\n-\t\t\t\t void *data);\n-int for_each_loose_file_in_objdir_buf(struct strbuf *path,\n-\t\t\t\t each_loose_object_fn obj_cb,\n-\t\t\t\t each_loose_cruft_fn cruft_cb,\n-\t\t\t\t each_loose_subdir_fn subdir_cb,\n-\t\t\t\t void *data);\n-\n-/* Flags for for_each_*_object() below. */\n+/* Flags for for_each_*_object(). */\n enum for_each_object_flags {\n \t/* Iterate only over local objects, not alternates. */\n \tFOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),\n@@ -409,33 +347,6 @@ enum for_each_object_flags {\n \tFOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),\n };\n \n-/*\n- * Iterate over all accessible loose objects without respect to\n- * reachability. By default, this includes both local and alternate objects.\n- * The order in which objects are visited is unspecified.\n- *\n- * Any flags specific to packs are ignored.\n- */\n-int for_each_loose_object(each_loose_object_fn, void *,\n-\t\t\t enum for_each_object_flags flags);\n-\n-/*\n- * Iterate over all accessible packed objects without respect to reachability.\n- * By default, this includes both local and alternate packs.\n- *\n- * Note that some objects may appear twice if they are found in multiple packs.\n- * Each pack is visited in an unspecified order. By default, objects within a\n- * pack are visited in pack-idx order (i.e., sorted by oid).\n- */\n-typedef int each_packed_object_fn(const struct object_id *oid,\n-\t\t\t\t struct packed_git *pack,\n-\t\t\t\t uint32_t pos,\n-\t\t\t\t void *data);\n-int for_each_object_in_pack(struct packed_git *p,\n-\t\t\t each_packed_object_fn, void *data,\n-\t\t\t enum for_each_object_flags flags);\n-int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,\n-\t\t\t void *data, enum for_each_object_flags flags);\n \n void *read_object_with_reference(struct repository *r,\n \t\t\t\t const struct object_id *oid,\ndiff --git a/packfile.h b/packfile.h\nindex 05499382397..3a3c77cf05a 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -3,6 +3,7 @@\n \n #include \"list.h\"\n #include \"object.h\"\n+#include \"object-store.h\"\n #include \"oidset.h\"\n \n /* in object-store.h */\n@@ -117,6 +118,24 @@ void for_each_file_in_pack_dir(const char *objdir,\n \t\t\t each_file_in_pack_dir_fn fn,\n \t\t\t void *data);\n \n+/*\n+ * Iterate over all accessible packed objects without respect to reachability.\n+ * By default, this includes both local and alternate packs.\n+ *\n+ * Note that some objects may appear twice if they are found in multiple packs.\n+ * Each pack is visited in an unspecified order. By default, objects within a\n+ * pack are visited in pack-idx order (i.e., sorted by oid).\n+ */\n+typedef int each_packed_object_fn(const struct object_id *oid,\n+\t\t\t\t struct packed_git *pack,\n+\t\t\t\t uint32_t pos,\n+\t\t\t\t void *data);\n+int for_each_object_in_pack(struct packed_git *p,\n+\t\t\t each_packed_object_fn, void *data,\n+\t\t\t enum for_each_object_flags flags);\n+int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,\n+\t\t\t void *data, enum for_each_object_flags flags);\n+\n /* A hook to report invalid files in pack directory */\n #define PACKDIR_FILE_PACK 1\n #define PACKDIR_FILE_IDX 2\ndiff --git a/prune-packed.c b/prune-packed.c\nindex c1d95a519d7..92fb4fbb0ed 100644\n--- a/prune-packed.c\n+++ b/prune-packed.c\n@@ -2,7 +2,7 @@\n \n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"packfile.h\"\n #include \"progress.h\"\n #include \"prune-packed.h\"\ndiff --git a/reachable.c b/reachable.c\nindex e5f56f40181..9dc748f0b9a 100644\n--- a/reachable.c\n+++ b/reachable.c\n@@ -14,7 +14,7 @@\n #include \"list-objects.h\"\n #include \"packfile.h\"\n #include \"worktree.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"pack-bitmap.h\"\n #include \"pack-mtimes.h\"\n #include \"config.h\"\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516743","messageId":"20250425-pks-object-store-cleanups-v2-5-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 05/13] object-store: allow fetching objects via `has_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:56Z","receivedAt":"2025-04-25T07:09:03Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"We're about to fully remove `repo_has_object_file()` in favor of\n`has_object()` given that the latter has better defaults: it neither\nreloads packfiles by default nor does it fetch any promised objects in\ncase they are missing.\n\nThe latter usecase keeps us from converting a couple of callsites that\ncurrently do fetch objects though. It is not really clear whether _all_\nof those callsites should be fetching objects, but for a subset of them\nit is the desired behaviour indeed.\n\nIntroduce a new flag `HAS_OBJECT_FETCH_PROMISOR` that causes the\nfunction to optionally fetch missing objects which are part of a\npromisor pack.\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.c | 9 ++++++---\n object-store.h | 10 +++++++---\n 2 files changed, 13 insertions(+), 6 deletions(-)\n\ndiff --git a/object-store.c b/object-store.c\nindex 0cbad5a19a0..0d873868a6d 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -937,12 +937,15 @@ void *read_object_with_reference(struct repository *r,\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags)\n {\n-\tint quick = !(flags & HAS_OBJECT_RECHECK_PACKED);\n-\tunsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |\n-\t\t(quick ? OBJECT_INFO_QUICK : 0);\n+\tunsigned object_info_flags = 0;\n \n \tif (!startup_info->have_repository)\n \t\treturn 0;\n+\tif (!(flags & HAS_OBJECT_RECHECK_PACKED))\n+\t\tobject_info_flags |= OBJECT_INFO_QUICK;\n+\tif (!(flags & HAS_OBJECT_FETCH_PROMISOR))\n+\t\tobject_info_flags |= OBJECT_INFO_SKIP_FETCH_OBJECT;\n+\n \treturn oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;\n }\n \ndiff --git a/object-store.h b/object-store.h\nindex 5bbdaba92d1..c6055376f49 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -266,12 +266,16 @@ int oid_object_info_extended(struct repository *r,\n \t\t\t const struct object_id *,\n \t\t\t struct object_info *, unsigned flags);\n \n-/* Retry packed storage after checking packed and loose storage */\n-#define HAS_OBJECT_RECHECK_PACKED 1\n+enum {\n+\t/* Retry packed storage after checking packed and loose storage */\n+\tHAS_OBJECT_RECHECK_PACKED = (1 << 0),\n+\t/* Allow fetching the object in case the repository has a promisor remote. */\n+\tHAS_OBJECT_FETCH_PROMISOR = (1 << 1),\n+};\n \n /*\n * Returns 1 if the object exists. This function will not lazily fetch objects\n- * in a partial clone.\n+ * in a partial clone by default.\n */\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags);\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516744","messageId":"20250425-pks-object-store-cleanups-v2-6-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 06/13] treewide: trivial conversions of `repo_has_object_file()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:57Z","receivedAt":"2025-04-25T07:09:04Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"As the comment of `repo_has_object_file()` and its `_with_flags()`\nvariant tells us, these functions are considered to be deprecated in\nfavor of `has_object()`. The benefit of the replacement is better\ndefaults: it doesn't fetch missing objects via promisor remotes, and\nneither does it reload packfiles if an object wasn't found by default.\n\nStart sunsetting the functions by replacing trivial callsites of it with\n`has_object()`:\n\n - `repo_has_object_file(...)` is equivalent to\n `has_object(..., HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)`.\n\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT)`\n is equivalent to `has_object(..., 0)`.\n\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_SKIP_FETCH_OBJECT)`\n is equivalent to `has_object(..., HAS_OBJECT_RECHECK_PACKED)`.\n\nWhile converting callsites to use `HAS_OBJECT_FETCH_PROMISOR` retains\nthe current behaviour, it is not entirely clear whether all of them\nreally should be fetching promised objects. A subset of such callsites\nwhere it is obvious that they really shouldn't fetch objects has been\nleft out of this commit and will be adapted over the next couple of\ncommits.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/cat-file.c | 3 ++-\n builtin/clone.c | 4 +---\n builtin/fetch.c | 15 +++++++--------\n builtin/receive-pack.c | 4 +++-\n builtin/remote.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n cache-tree.c | 13 +++++++++----\n fetch-pack.c | 7 +++----\n http-push.c | 11 +++++++----\n notes.c | 3 ++-\n object-store.c | 2 +-\n reflog.c | 3 ++-\n remote.c | 2 +-\n send-pack.c | 5 +----\n shallow.c | 9 ++++++---\n upload-pack.c | 3 +--\n walker.c | 3 ++-\n 17 files changed, 52 insertions(+), 41 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 0e3f10a9467..3914a2a3f61 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -169,7 +169,8 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name,\n \t\tgoto cleanup;\n \n \tcase 'e':\n-\t\tret = !repo_has_object_file(the_repository, &oid);\n+\t\tret = !has_object(the_repository, &oid,\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR);\n \t\tgoto cleanup;\n \n \tcase 'w':\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 6b1d11a3ed2..b498b81a043 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -504,9 +504,7 @@ static void write_followtags(const struct ref *refs, const char *msg)\n \t\t\tcontinue;\n \t\tif (ends_with(ref->name, \"^{}\"))\n \t\t\tcontinue;\n-\t\tif (!repo_has_object_file_with_flags(the_repository, &ref->old_oid,\n-\t\t\t\t\t\t OBJECT_INFO_QUICK |\n-\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (!has_object(the_repository, &ref->old_oid, 0))\n \t\t\tcontinue;\n \t\trefs_update_ref(get_main_ref_store(the_repository), msg,\n \t\t\t\tref->name, &ref->old_oid, NULL, 0,\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 95589b49948..aadcf49a5b4 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -337,7 +337,6 @@ static void find_non_local_tags(const struct ref *refs,\n \tstruct string_list_item *remote_ref_item;\n \tconst struct ref *ref;\n \tstruct refname_hash_entry *item = NULL;\n-\tconst int quick_flags = OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT;\n \n \trefname_hash_init(&existing_refs);\n \trefname_hash_init(&remote_refs);\n@@ -367,9 +366,9 @@ static void find_non_local_tags(const struct ref *refs,\n \t\t */\n \t\tif (ends_with(ref->name, \"^{}\")) {\n \t\t\tif (item &&\n-\t\t\t !repo_has_object_file_with_flags(the_repository, &ref->old_oid, quick_flags) &&\n+\t\t\t !has_object(the_repository, &ref->old_oid, 0) &&\n \t\t\t !oidset_contains(&fetch_oids, &ref->old_oid) &&\n-\t\t\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t\t\t !has_object(the_repository, &item->oid, 0) &&\n \t\t\t !oidset_contains(&fetch_oids, &item->oid))\n \t\t\t\tclear_item(item);\n \t\t\titem = NULL;\n@@ -383,7 +382,7 @@ static void find_non_local_tags(const struct ref *refs,\n \t\t * fetch.\n \t\t */\n \t\tif (item &&\n-\t\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t\t !has_object(the_repository, &item->oid, 0) &&\n \t\t !oidset_contains(&fetch_oids, &item->oid))\n \t\t\tclear_item(item);\n \n@@ -404,7 +403,7 @@ static void find_non_local_tags(const struct ref *refs,\n \t * checked to see if it needs fetching.\n \t */\n \tif (item &&\n-\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t !has_object(the_repository, &item->oid, 0) &&\n \t !oidset_contains(&fetch_oids, &item->oid))\n \t\tclear_item(item);\n \n@@ -911,7 +910,8 @@ static int update_local_ref(struct ref *ref,\n \tstruct commit *current = NULL, *updated;\n \tint fast_forward = 0;\n \n-\tif (!repo_has_object_file(the_repository, &ref->new_oid))\n+\tif (!has_object(the_repository, &ref->new_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\tdie(_(\"object %s not found\"), oid_to_hex(&ref->new_oid));\n \n \tif (oideq(&ref->old_oid, &ref->new_oid)) {\n@@ -1330,8 +1330,7 @@ static int check_exist_and_connected(struct ref *ref_map)\n \t * we need all direct targets to exist.\n \t */\n \tfor (r = rm; r; r = r->next) {\n-\t\tif (!repo_has_object_file_with_flags(the_repository, &r->old_oid,\n-\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (!has_object(the_repository, &r->old_oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\treturn -1;\n \t}\n \ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex be314879e82..c92e57ba188 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -1506,7 +1506,9 @@ static const char *update(struct command *cmd, struct shallow_info *si)\n \t\t}\n \t}\n \n-\tif (!is_null_oid(new_oid) && !repo_has_object_file(the_repository, new_oid)) {\n+\tif (!is_null_oid(new_oid) &&\n+\t !has_object(the_repository, new_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\terror(\"unpack should have generated %s, \"\n \t\t \"but I can't find it!\", oid_to_hex(new_oid));\n \t\tret = \"bad pack\";\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex b4baa34e665..0d6755bcb71 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -454,7 +454,8 @@ static int get_push_ref_states(const struct ref *remote_refs,\n \t\t\tinfo->status = PUSH_STATUS_UPTODATE;\n \t\telse if (is_null_oid(&ref->old_oid))\n \t\t\tinfo->status = PUSH_STATUS_CREATE;\n-\t\telse if (repo_has_object_file(the_repository, &ref->old_oid) &&\n+\t\telse if (has_object(the_repository, &ref->old_oid,\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) &&\n \t\t\t ref_newer(&ref->new_oid, &ref->old_oid))\n \t\t\tinfo->status = PUSH_STATUS_FASTFORWARD;\n \t\telse\ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex 661be789f13..e905d5f4e19 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -449,7 +449,8 @@ static void unpack_delta_entry(enum object_type type, unsigned long delta_size,\n \t\tdelta_data = get_data(delta_size);\n \t\tif (!delta_data)\n \t\t\treturn;\n-\t\tif (repo_has_object_file(the_repository, &base_oid))\n+\t\tif (has_object(the_repository, &base_oid,\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\t; /* Ok we have this one */\n \t\telse if (resolve_against_held(nr, &base_oid,\n \t\t\t\t\t delta_data, delta_size))\ndiff --git a/cache-tree.c b/cache-tree.c\nindex c0e1e9ee1d4..fa3858e2829 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -238,7 +238,9 @@ int cache_tree_fully_valid(struct cache_tree *it)\n \tint i;\n \tif (!it)\n \t\treturn 0;\n-\tif (it->entry_count < 0 || !repo_has_object_file(the_repository, &it->oid))\n+\tif (it->entry_count < 0 ||\n+\t has_object(the_repository, &it->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn 0;\n \tfor (i = 0; i < it->subtree_nr; i++) {\n \t\tif (!cache_tree_fully_valid(it->down[i]->cache_tree))\n@@ -289,7 +291,9 @@ static int update_one(struct cache_tree *it,\n \t\t}\n \t}\n \n-\tif (0 <= it->entry_count && repo_has_object_file(the_repository, &it->oid))\n+\tif (0 <= it->entry_count &&\n+\t has_object(the_repository, &it->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn it->entry_count;\n \n \t/*\n@@ -395,7 +399,8 @@ static int update_one(struct cache_tree *it,\n \t\tce_missing_ok = mode == S_IFGITLINK || missing_ok ||\n \t\t\t!must_check_existence(ce);\n \t\tif (is_null_oid(oid) ||\n-\t\t (!ce_missing_ok && !repo_has_object_file(the_repository, oid))) {\n+\t\t (!ce_missing_ok && !has_object(the_repository, oid,\n+\t\t\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))) {\n \t\t\tstrbuf_release(&buffer);\n \t\t\tif (expected_missing)\n \t\t\t\treturn -1;\n@@ -443,7 +448,7 @@ static int update_one(struct cache_tree *it,\n \t\tstruct object_id oid;\n \t\thash_object_file(the_hash_algo, buffer.buf, buffer.len,\n \t\t\t\t OBJ_TREE, &oid);\n-\t\tif (repo_has_object_file_with_flags(the_repository, &oid, OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (has_object(the_repository, &oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\toidcpy(&it->oid, &oid);\n \t\telse\n \t\t\tto_invalidate = 1;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 210dc30d50f..fa4231fee74 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -769,9 +769,7 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator,\n \t\tif (!commit) {\n \t\t\tstruct object *o;\n \n-\t\t\tif (!repo_has_object_file_with_flags(the_repository, &ref->old_oid,\n-\t\t\t\t\t\t\t OBJECT_INFO_QUICK |\n-\t\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\t\tif (!has_object(the_repository, &ref->old_oid, 0))\n \t\t\t\tcontinue;\n \t\t\to = parse_object(the_repository, &ref->old_oid);\n \t\t\tif (!o || o->type != OBJ_COMMIT)\n@@ -1985,7 +1983,8 @@ static void update_shallow(struct fetch_pack_args *args,\n \t\tstruct oid_array extra = OID_ARRAY_INIT;\n \t\tstruct object_id *oid = si->shallow->oid;\n \t\tfor (i = 0; i < si->shallow->nr; i++)\n-\t\t\tif (repo_has_object_file(the_repository, &oid[i]))\n+\t\t\tif (has_object(the_repository, &oid[i],\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\t\toid_array_append(&extra, &oid[i]);\n \t\tif (extra.nr) {\n \t\t\tsetup_alternate_shallow(&shallow_lock,\ndiff --git a/http-push.c b/http-push.c\nindex 32e37565f4e..f9e67cabd4b 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -1446,7 +1446,9 @@ static void one_remote_ref(const char *refname)\n \t * Fetch a copy of the object if it doesn't exist locally - it\n \t * may be required for updating server info later.\n \t */\n-\tif (repo->can_update_info_refs && !repo_has_object_file(the_repository, &ref->old_oid)) {\n+\tif (repo->can_update_info_refs &&\n+\t !has_object(the_repository, &ref->old_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\tobj = lookup_unknown_object(the_repository, &ref->old_oid);\n \t\tfprintf(stderr,\t\" fetch %s for %s\\n\",\n \t\t\toid_to_hex(&ref->old_oid), refname);\n@@ -1651,14 +1653,14 @@ static int delete_remote_branch(const char *pattern, int force)\n \t\t\treturn error(\"Remote HEAD symrefs too deep\");\n \t\tif (is_null_oid(&head_oid))\n \t\t\treturn error(\"Unable to resolve remote HEAD\");\n-\t\tif (!repo_has_object_file(the_repository, &head_oid))\n+\t\tif (!has_object(the_repository, &head_oid, HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn error(\"Remote HEAD resolves to object %s\\nwhich does not exist locally, perhaps you need to fetch?\", oid_to_hex(&head_oid));\n \n \t\t/* Remote branch must resolve to a known object */\n \t\tif (is_null_oid(&remote_ref->old_oid))\n \t\t\treturn error(\"Unable to resolve remote branch %s\",\n \t\t\t\t remote_ref->name);\n-\t\tif (!repo_has_object_file(the_repository, &remote_ref->old_oid))\n+\t\tif (!has_object(the_repository, &remote_ref->old_oid, HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn error(\"Remote branch %s resolves to object %s\\nwhich does not exist locally, perhaps you need to fetch?\", remote_ref->name, oid_to_hex(&remote_ref->old_oid));\n \n \t\t/* Remote branch must be an ancestor of remote HEAD */\n@@ -1879,7 +1881,8 @@ int cmd_main(int argc, const char **argv)\n \t\tif (!force_all &&\n \t\t !is_null_oid(&ref->old_oid) &&\n \t\t !ref->force) {\n-\t\t\tif (!repo_has_object_file(the_repository, &ref->old_oid) ||\n+\t\t\tif (!has_object(the_repository, &ref->old_oid,\n+\t\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) ||\n \t\t\t !ref_newer(&ref->peer_ref->new_oid,\n \t\t\t\t &ref->old_oid)) {\n \t\t\t\t/*\ndiff --git a/notes.c b/notes.c\nindex d9645c4b5dc..0a128f1de98 100644\n--- a/notes.c\n+++ b/notes.c\n@@ -794,7 +794,8 @@ static int prune_notes_helper(const struct object_id *object_oid,\n \tstruct note_delete_list **l = (struct note_delete_list **) cb_data;\n \tstruct note_delete_list *n;\n \n-\tif (repo_has_object_file(the_repository, object_oid))\n+\tif (has_object(the_repository, object_oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn 0; /* nothing to do for this note */\n \n \t/* failed to find object => prune this note */\ndiff --git a/object-store.c b/object-store.c\nindex 0d873868a6d..2db34804e8f 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -847,7 +847,7 @@ int pretend_object_file(struct repository *repo,\n \tchar *co_buf;\n \n \thash_object_file(repo->hash_algo, buf, len, type, oid);\n-\tif (repo_has_object_file_with_flags(repo, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||\n+\tif (has_object(repo, oid, 0) ||\n \t find_cached_object(repo->objects, oid))\n \t\treturn 0;\n \ndiff --git a/reflog.c b/reflog.c\nindex c6c534dfad4..2db1cc81d01 100644\n--- a/reflog.c\n+++ b/reflog.c\n@@ -152,7 +152,8 @@ static int tree_is_complete(const struct object_id *oid)\n \tinit_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size);\n \tcomplete = 1;\n \twhile (tree_entry(&desc, &entry)) {\n-\t\tif (!repo_has_object_file(the_repository, &entry.oid) ||\n+\t\tif (!has_object(the_repository, &entry.oid,\n+\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) ||\n \t\t (S_ISDIR(entry.mode) && !tree_is_complete(&entry.oid))) {\n \t\t\ttree->object.flags |= INCOMPLETE;\n \t\t\tcomplete = 0;\ndiff --git a/remote.c b/remote.c\nindex 9fa3614e7a3..4099183cacd 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -1702,7 +1702,7 @@ void set_ref_status_for_push(struct ref *remote_refs, int send_mirror,\n \t\tif (!reject_reason && !ref->deletion && !is_null_oid(&ref->old_oid)) {\n \t\t\tif (starts_with(ref->name, \"refs/tags/\"))\n \t\t\t\treject_reason = REF_STATUS_REJECT_ALREADY_EXISTS;\n-\t\t\telse if (!repo_has_object_file_with_flags(the_repository, &ref->old_oid, OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\t\telse if (!has_object(the_repository, &ref->old_oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\t\treject_reason = REF_STATUS_REJECT_FETCH_FIRST;\n \t\t\telse if (!lookup_commit_reference_gently(the_repository, &ref->old_oid, 1) ||\n \t\t\t\t !lookup_commit_reference_gently(the_repository, &ref->new_oid, 1))\ndiff --git a/send-pack.c b/send-pack.c\nindex 5005689cb55..86592ce526d 100644\n--- a/send-pack.c\n+++ b/send-pack.c\n@@ -45,10 +45,7 @@ int option_parse_push_signed(const struct option *opt,\n static void feed_object(struct repository *r,\n \t\t\tconst struct object_id *oid, FILE *fh, int negative)\n {\n-\tif (negative &&\n-\t !repo_has_object_file_with_flags(r, oid,\n-\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT |\n-\t\t\t\t\t OBJECT_INFO_QUICK))\n+\tif (negative && !has_object(r, oid, 0))\n \t\treturn;\n \n \tif (negative)\ndiff --git a/shallow.c b/shallow.c\nindex 2f82ebd6e3f..faeeeb45f98 100644\n--- a/shallow.c\n+++ b/shallow.c\n@@ -310,7 +310,8 @@ static int write_one_shallow(const struct commit_graft *graft, void *cb_data)\n \tif (graft->nr_parent != -1)\n \t\treturn 0;\n \tif (data->flags & QUICK) {\n-\t\tif (!repo_has_object_file(the_repository, &graft->oid))\n+\t\tif (!has_object(the_repository, &graft->oid,\n+\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn 0;\n \t} else if (data->flags & SEEN_ONLY) {\n \t\tstruct commit *c = lookup_commit(the_repository, &graft->oid);\n@@ -476,7 +477,8 @@ void prepare_shallow_info(struct shallow_info *info, struct oid_array *sa)\n \tALLOC_ARRAY(info->ours, sa->nr);\n \tALLOC_ARRAY(info->theirs, sa->nr);\n \tfor (size_t i = 0; i < sa->nr; i++) {\n-\t\tif (repo_has_object_file(the_repository, sa->oid + i)) {\n+\t\tif (has_object(the_repository, sa->oid + i,\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\t\tstruct commit_graft *graft;\n \t\t\tgraft = lookup_commit_graft(the_repository,\n \t\t\t\t\t\t &sa->oid[i]);\n@@ -513,7 +515,8 @@ void remove_nonexistent_theirs_shallow(struct shallow_info *info)\n \tfor (i = dst = 0; i < info->nr_theirs; i++) {\n \t\tif (i != dst)\n \t\t\tinfo->theirs[dst] = info->theirs[i];\n-\t\tif (repo_has_object_file(the_repository, oid + info->theirs[i]))\n+\t\tif (has_object(the_repository, oid + info->theirs[i],\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\tdst++;\n \t}\n \tinfo->nr_theirs = dst;\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 30e4630f3a1..956da5b061a 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -509,8 +509,7 @@ static int got_oid(struct upload_pack_data *data,\n {\n \tif (get_oid_hex(hex, oid))\n \t\tdie(\"git upload-pack: expected SHA1 object, got '%s'\", hex);\n-\tif (!repo_has_object_file_with_flags(the_repository, oid,\n-\t\t\t\t\t OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\tif (!has_object(the_repository, oid, 0))\n \t\treturn -1;\n \treturn do_got_oid(data, oid);\n }\ndiff --git a/walker.c b/walker.c\nindex 4fedc19f346..b470d43e54d 100644\n--- a/walker.c\n+++ b/walker.c\n@@ -150,7 +150,8 @@ static int process(struct walker *walker, struct object *obj)\n \t\treturn 0;\n \tobj->flags |= SEEN;\n \n-\tif (repo_has_object_file(the_repository, &obj->oid)) {\n+\tif (has_object(the_repository, &obj->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\t/* We already have it, so we should scan it now. */\n \t\tobj->flags |= TO_SCAN;\n \t}\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516745","messageId":"20250425-pks-object-store-cleanups-v2-7-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 07/13] builtin/index-pack: don't fetch promised objects for collision check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:58Z","receivedAt":"2025-04-25T07:09:05Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Any packed objects indexed via git-index-pack(1) are subject to a\ncollision check. This collision check has the intent to determine\nwhether we already have an object with the same object ID, but different\ncontents in the repository.\n\nThe check whether the collision check is really needed is only performed\nin case `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)` tells\nus that the object exists. But unless explicitly told otherwise by\npassing `OBJECT_INFO_SKIP_FETCH_OBJECT`, this function will also cause\nus to fetch the object in case it is part of a promisor pack. As such,\nwe may end up fetching the object only to check whether the fetched\nobject and the object that we're indexing have the same content.\n\nThis behaviour is highly dubious and more likely than not unintended.\nFix it by converting to `has_object()`, which knows to neither reload\npackfiles nor to fetch promisor objects by default.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/index-pack.c | 4 +---\n 1 file changed, 1 insertion(+), 3 deletions(-)\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex f49431d626b..805b7aa1e28 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -892,9 +892,7 @@ static void sha1_object(const void *data, struct object_entry *obj_entry,\n \n \tif (startup_info->have_repository) {\n \t\tread_lock();\n-\t\tcollision_test_needed =\n-\t\t\trepo_has_object_file_with_flags(the_repository, oid,\n-\t\t\t\t\t\t\tOBJECT_INFO_QUICK);\n+\t\tcollision_test_needed = has_object(the_repository, oid, 0);\n \t\tread_unlock();\n \t}\n \n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516746","messageId":"20250425-pks-object-store-cleanups-v2-8-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 08/13] builtin/show-ref: don't fetch objects when printing refs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:08:59Z","receivedAt":"2025-04-25T07:09:06Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When printing references via git-show-ref(1) we double check that their\nrespective object IDs point to an existing object. This check is\nperformed via `repo_has_object_file()`, which knows to fetch missing\npromised objects in the background. We shouldn't have a need to fetch\nsuch objects though as no reference should ever point to a missing\nobject at all.\n\nConvert the callsite to `has_object()`, which doesn't fetch promisor\nobjects by default.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/show-ref.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/builtin/show-ref.c b/builtin/show-ref.c\nindex f81209f23c3..fe592b4c202 100644\n--- a/builtin/show-ref.c\n+++ b/builtin/show-ref.c\n@@ -35,7 +35,7 @@ static void show_one(const struct show_one_options *opts,\n \tconst char *hex;\n \tstruct object_id peeled;\n \n-\tif (!repo_has_object_file(the_repository, oid))\n+\tif (!has_object(the_repository, oid, HAS_OBJECT_RECHECK_PACKED))\n \t\tdie(\"git show-ref: bad ref %s (%s)\", refname,\n \t\t oid_to_hex(oid));\n \n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516747","messageId":"20250425-pks-object-store-cleanups-v2-9-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 09/13] refs: don't fetch promisor objects in `ref_resolves_to_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:09:00Z","receivedAt":"2025-04-25T07:09:07Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Similar to the preceding commit, don't try to fetch objects pointed to\nby references. Any reference whose object does not exist is broken by\ndefinition, so we should report it accordingly.\n\nSigned-off-by: Patrick Steinhardt \n---\n refs.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/refs.c b/refs.c\nindex 6559db37890..0492cf0d13a 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -376,7 +376,7 @@ int ref_resolves_to_object(const char *refname,\n {\n \tif (flags & REF_ISBROKEN)\n \t\treturn 0;\n-\tif (!repo_has_object_file(repo, oid)) {\n+\tif (!has_object(repo, oid, HAS_OBJECT_RECHECK_PACKED)) {\n \t\terror(_(\"%s does not point to a valid object!\"), refname);\n \t\treturn 0;\n \t}\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516748","messageId":"20250425-pks-object-store-cleanups-v2-10-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 10/13] http-walker: don't fetch objects via promisor remotes","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:09:01Z","receivedAt":"2025-04-25T07:09:08Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The HTTP walker is responsible for fetching objects via the dumb HTTP\nprotocol. To avoid re-fetching objects that we already have locally we\nfirst check whether the object already exists in the local repository\nbefore issuing the requests.\n\nThis existence check is done by calling `repo_has_object_file()`, which\nwill fetch the object via a promisor remote in case it is in a promisor\npack. This fetch does not make any sense for us though: we're already in\nthe process of fetching the object anyway, so fetching it via a separate\nconnection is wasteful, but should otherwise be harmless.\n\nFix the issue by converting to `has_object()`, which knows to not fetch\nobjects via promisor remotes by default.\n\nSigned-off-by: Patrick Steinhardt \n---\n http-walker.c | 4 ++--\n 1 file changed, 2 insertions(+), 2 deletions(-)\n\ndiff --git a/http-walker.c b/http-walker.c\nindex 95458e2f638..5ad2eae9a11 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -138,7 +138,7 @@ static int fill_active_slot(void *data UNUSED)\n \tlist_for_each_safe(pos, tmp, head) {\n \t\tobj_req = list_entry(pos, struct object_request, node);\n \t\tif (obj_req->state == WAITING) {\n-\t\t\tif (repo_has_object_file(the_repository, &obj_req->oid))\n+\t\t\tif (has_object(the_repository, &obj_req->oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\t\tobj_req->state = COMPLETE;\n \t\t\telse {\n \t\t\t\tstart_object_request(obj_req);\n@@ -496,7 +496,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)\n \tif (!obj_req)\n \t\treturn error(\"Couldn't find request for %s in the queue\", hex);\n \n-\tif (repo_has_object_file(the_repository, &obj_req->oid)) {\n+\tif (has_object(the_repository, &obj_req->oid, HAS_OBJECT_RECHECK_PACKED)) {\n \t\tif (obj_req->req)\n \t\t\tabort_http_object_request(&obj_req->req);\n \t\tabort_object_request(obj_req);\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516749","messageId":"20250425-pks-object-store-cleanups-v2-11-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 11/13] list-objects: clarify how promised blobs are excluded","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:09:02Z","receivedAt":"2025-04-25T07:09:09Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When \"--exclude-promisor-objects\" is passed by the user, commands like\ngit-rev-list(1) will exclude any objects part of a promisor pack. For\nblobs this logic is handled in `process_blob()` with the following\nlogic:\n\n if (ctx->revs->exclude_promisor_objects &&\n !repo_has_object_file(the_repository, &obj->oid) &&\n is_promisor_object(ctx->revs->repo, &obj->oid))\n return;\n\nIt is somewhat puzzling that we use `repo_has_object_file()` to check\nfor existence of the blob because this function will cause us to fetch\nmissing objects in case they are part of a promisor pack. As such, one\nmay wonder whether the logic to exclude promised blobs is completely\nbroken.\n\nAs it turns out it's not broken: when \"--exclude-promisor-objects\" is\nset we also unset the global `fetch_if_missing` variable, which causes\n`do_oid_object_info_extended()` to not fetch any promised objects at\nall.\n\nClarify this logic by using `has_object()`, which doesn't fetch promised\nobjects by default.\n\nSigned-off-by: Patrick Steinhardt \n---\n list-objects.c | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)\n\ndiff --git a/list-objects.c b/list-objects.c\nindex 1e5512e1318..cae4f7aff8c 100644\n--- a/list-objects.c\n+++ b/list-objects.c\n@@ -74,7 +74,8 @@ static void process_blob(struct traversal_context *ctx,\n \t * of missing objects.\n \t */\n \tif (ctx->revs->exclude_promisor_objects &&\n-\t !repo_has_object_file(the_repository, &obj->oid) &&\n+\t !has_object(the_repository, &obj->oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED) &&\n \t is_promisor_object(ctx->revs->repo, &obj->oid))\n \t\treturn;\n \n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516750","messageId":"20250425-pks-object-store-cleanups-v2-12-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 12/13] bulk-checkin: don't fetch promised objects on write","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:09:03Z","receivedAt":"2025-04-25T07:09:09Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When writing objects via the bulk-checkin subsystem we first try to\nfigure out whether an object already exists in the repository before we\nappend it to the packfile. This check uses `repo_has_object_file()`,\nwhich knows to fetch promised objects by default. As such, if we were\nabout to write an object that is promised, we'd fetch the object via the\npromisor and then skip writing it. This behaviour doesn't seem sensible:\nit should be significantly faster to take the locally-written object\ninstead of faulting in objects from the promisor remote.\n\nThere is one counter-argument here: it could be that the bulk-checkin\nmechanism will end up writing an object to disk whose content collides\nwith the object in the promisor remote. The local repository and its\npromisor remote would now have two objects with different contents but\nthe same name. But the resulting behaviour would be wrong both when we\nprefer the fetched object, and also when prefering the written object:\n\n - When we prefer the written object we will now see a different world\n compared to everyone else who has the promised object.\n\n - When we prefer the fetched object we will end up with an object that\n is different compared to what the user just asked us to write. This\n seems even worse compared to the first scenario.\n\nIn an ideal world, we would protect against this by fetching the\npromised object and then performing a collision check. But this feels\nexceedingly expensive and ultimately rather pointless, as more common\nwriting paths like `write_loose_object()` don't protect against this\nscenario either. And in any case we're talking about a local user that\nhas write access to the repository anyway, so if they want to do any\nkind of mischieve they already can.\n\nChange the behaviour so that we don't fault in the object via the\npromisor remote. We shouldn't have to worry about hash collisions too\nmuch (yet) as the mechanism is only used during local writes anyway. And\neven if there was a collision, prefering local data that we were just\nasked to write over data controlled by a potentially untrusted remote\nfeels like the better failure mode.\n\nSigned-off-by: Patrick Steinhardt \n---\n bulk-checkin.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/bulk-checkin.c b/bulk-checkin.c\nindex c31c31b18d8..b182c456d69 100644\n--- a/bulk-checkin.c\n+++ b/bulk-checkin.c\n@@ -130,7 +130,7 @@ static void flush_batch_fsync(void)\n static int already_written(struct bulk_checkin_packfile *state, struct object_id *oid)\n {\n \t/* The object may already exist in the repository */\n-\tif (repo_has_object_file(the_repository, oid))\n+\tif (has_object(the_repository, oid, HAS_OBJECT_RECHECK_PACKED))\n \t\treturn 1;\n \n \t/* Might want to keep the list sorted */\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516751","messageId":"20250425-pks-object-store-cleanups-v2-13-63f1695b7700@pks.im","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"[PATCH v2 13/13] object-store: drop `repo_has_object_file()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-25T07:09:04Z","receivedAt":"2025-04-25T07:09:10Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In the preceding commits we have converted all users of\n`repo_has_object_file()` and its `_with_flags()` variant to instead use\n`has_object()`. Drop these functions.\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.c | 14 --------------\n object-store.h | 17 -----------------\n 2 files changed, 31 deletions(-)\n\ndiff --git a/object-store.c b/object-store.c\nindex 2db34804e8f..2f51d0e3b03 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -949,20 +949,6 @@ int has_object(struct repository *r, const struct object_id *oid,\n \treturn oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;\n }\n \n-int repo_has_object_file_with_flags(struct repository *r,\n-\t\t\t\t const struct object_id *oid, int flags)\n-{\n-\tif (!startup_info->have_repository)\n-\t\treturn 0;\n-\treturn oid_object_info_extended(r, oid, NULL, flags) >= 0;\n-}\n-\n-int repo_has_object_file(struct repository *r,\n-\t\t\t const struct object_id *oid)\n-{\n-\treturn repo_has_object_file_with_flags(r, oid, 0);\n-}\n-\n void assert_oid_type(const struct object_id *oid, enum object_type expect)\n {\n \tenum object_type type = oid_object_info(the_repository, oid, NULL);\ndiff --git a/object-store.h b/object-store.h\nindex c6055376f49..2330374990b 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -280,23 +280,6 @@ enum {\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags);\n \n-/*\n- * These macros and functions are deprecated. If checking existence for an\n- * object that is likely to be missing and/or whose absence is relatively\n- * inconsequential (or is consequential but the caller is prepared to handle\n- * it), use has_object(), which has better defaults (no lazy fetch in a partial\n- * clone and no rechecking of packed storage). In the unlikely event that a\n- * caller needs to assert existence of an object that it fully expects to\n- * exist, and wants to trigger a lazy fetch in a partial clone, use\n- * oid_object_info_extended() with a NULL struct object_info.\n- *\n- * These functions can be removed once all callers have migrated to\n- * has_object() and/or oid_object_info_extended().\n- */\n-int repo_has_object_file(struct repository *r, const struct object_id *oid);\n-int repo_has_object_file_with_flags(struct repository *r,\n-\t\t\t\t const struct object_id *oid, int flags);\n-\n void assert_oid_type(const struct object_id *oid, enum object_type expect);\n \n /*\n\n-- \n2.49.0.901.g37484f566f.dirty\n\n"},{"id":"516908","messageId":"CAOLa=ZQy=OrWkqUTS+t_fAYj2w--1Fy=n7=gdJTHFhebNAZCnA@mail.gmail.com","threadId":"63335","inReplyTo":"aAs0F58Hlpn6WI6b@pks.im","subject":"Re: [PATCH 07/13] builtin/index-pack: don't fetch promised objects for collision check","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-28T19:48:23Z","receivedAt":"2025-04-28T19:48:25Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> On Wed, Apr 23, 2025 at 10:08:05AM -0700, Karthik Nayak wrote:\n>> Patrick Steinhardt writes:\n>>\n>> > Any packed objects indexed via git-index-pack(1) are subject to a\n>> > collision check. This collision check has the intent to determine\n>> > whether we already have an object with the same object ID, but different\n>> > contents in the repository.\n>> >\n>> > The check whether the collision check is really needed is performed via\n>> > `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)`. \\\n>> >\n>>\n>> Nit: this was a little confusing at first, until I saw the code. So what\n>> this means is that the collision check is only performed, iff\n>> `repo_has_object_file_with_flags(...)` returns true.\n>>\n>> I think the confusing part was 'is performed via', perhaps:\n>>\n>> The collision check is only performed, if\n>> repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK) returns a\n>> truthy value.\n>>\n>> But it is okay as is too!\n>\n> Will rephrase.\n>\n>> > But unless\n>> > explicitly told otherwise via `OBJECT_INFO_SKIP_FETCH_OBJECT`, this\n>> > function will also cause us to fetch the object ID in case it is part of\n>> > a promisor pack. As such, we may end up fetching the object only to\n>> > check whether the fetched object and the object that we're indexing have\n>> > the same content.\n>> >\n>>\n>> So us fetching the object is pointless, since we only care about the\n>> 'does it exist' part and not really what it contains. In that case,\n>> shouldn't this be s/same content/same oid/?\n>\n> No, it really checks for the same content. It basically verifies that\n> any pair of objects that:\n>\n> - Exist in the packfile that we're currently indexing.\n> - And preexists in the local repository.\n>\n> Actually have the same content.\n>\n\nOkay this makes sense, if they do have the same content, this is not\na collision. It is simply a duplicate.\n\n> The weird part is that we also do this for objects that don't yet exist\n> in the repository, but which are promised to us. This causes us to fetch\n> them first only to verify that the fetched promised object has the same\n> content as the packfile. And given that git-index-pack(1) would usually\n> run after a fetch, we end up verifying that the fetched object obtained\n> from the promisor is the same as the fetched object obtained from the\n> packfile. Which ultimately seems rather dubious to me.\n>\n\nTo clarify, the flow currently (simplified) is:\n\n1. We check if a collision test is required, by checking if the new OID\nalready exists in the repository.\n2. If collision test is required.\n a. Fetch and check the object type.\n b. Read the old object data.\n c. Compare the new object data and the old object data.\n d. Collision detected if there is a mismatch.\n\nCurrently, we fetch for promisor objects in #1, which is unnecessary\nbecause we simply want to know if the object exists in the repository.\nThe actual check in #2.b would still fetch the promisor object (if that\nflow is taken).\n\n> Patrick\n"},{"id":"516909","messageId":"CAOLa=ZSPoWOpMPj3kD5=aZQfU_JGX1QH_frJ71stPrtK_VQ4RQ@mail.gmail.com","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im","subject":"Re: [PATCH v2 00/13] object-store: a handful of cleanups","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-04-28T19:49:29Z","receivedAt":"2025-04-28T19:49:31Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> Hi,\n>\n> this patch series contains a handful of cleanups to the object store\n> subsystem:\n>\n> - A couple of definitions are moved out of \"object-store.h\" as they\n> belong to other subsystems.\n>\n> - Some functions are dropped and/or renamed.\n>\n> - The biggest part is the removal of `repo_has_object_file()`. This\n> function and its `_with_flags()` variant are marked as deprecated,\n> with the replacement being `has_object()`. The benefit of that\n> function is that it doesn't reload packfiles and doesn't fetch\n> promisor objects by default so that it becomes more explicit when\n> one really wants to do so.\n>\n> These cleanups are in preparation for getting rid of `the_repository` in\n> \"object-store.c\".\n>\n> The patch series is built on top of 4bbb303af69 (The seventh batch,\n> 2025-04-17) with ps/object-file-cleanup at 68cd492a3e6 (object-store:\n> merge \"object-store-ll.h\" and \"object-store.h\", 2025-04-15) merged into\n> it.\n>\n> Changes in v2:\n> - A handful of improvements for commit messages.\n> - Link to v1: https://lore.kernel.org/r/20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im\n>\n> Thanks!\n>\n> Patrick\n\n[snip]\n\nThe range-diff looked good to me. Thanks!\n"},{"id":"516918","messageId":"xmqq7c34arsk.fsf@gitster.g","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-6-63f1695b7700@pks.im","subject":"Re: [PATCH v2 06/13] treewide: trivial conversions of `repo_has_object_file()`","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-28T21:40:27Z","receivedAt":"2025-04-28T21:40:29Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> As the comment of `repo_has_object_file()` and its `_with_flags()`\n> variant tells us, these functions are considered to be deprecated in\n> favor of `has_object()`. The benefit of the replacement is better\n> defaults: it doesn't fetch missing objects via promisor remotes, and\n> neither does it reload packfiles if an object wasn't found by default.\n\nThe reason why the \"better default\" being \"better\" is not clear to\nreaders of this paragraph.\n\nIt would be better if too many places need to pass an extra flag to\ndisable lazy fetching in the current code, for example, in which\ncase we would instead have fewer very selected places that would\npass an extra flag to lazily fetch (or even better, see has_object()\nfail and then invoke a lazy fetch themselves) while majority of\nplaces would just work with the plain vanilla local repository state\nwithout having to worry about lazy fetching at all. Without\nexplaining why it is a \"better default\", it is a bit hard to justify\n\"the benefit of the replacement\".\n\nThis is common to this and the previous step.\n"},{"id":"516919","messageId":"xmqq34dsarhv.fsf@gitster.g","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-7-63f1695b7700@pks.im","subject":"Re: [PATCH v2 07/13] builtin/index-pack: don't fetch promised objects for collision check","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-28T21:46:52Z","receivedAt":"2025-04-28T21:46:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Any packed objects indexed via git-index-pack(1) are subject to a\n> collision check. This collision check has the intent to determine\n> whether we already have an object with the same object ID, but different\n> contents in the repository.\n>\n> The check whether the collision check is really needed is only performed\n> in case `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)` tells\n> us that the object exists. But unless explicitly told otherwise by\n> passing `OBJECT_INFO_SKIP_FETCH_OBJECT`, this function will also cause\n> us to fetch the object in case it is part of a promisor pack. As such,\n> we may end up fetching the object only to check whether the fetched\n> object and the object that we're indexing have the same content.\n>\n> This behaviour is highly dubious and more likely than not unintended.\n> Fix it by converting to `has_object()`, which knows to neither reload\n> packfiles nor to fetch promisor objects by default.\n\nIt is unclear why you thing it is highly dubious from reading the\nabove paragraph three times, though.\n\nIs it that if we are suspicious of the incoming pack data we are\nindexing, we should also not be too trusting of the object that our\npromisor remote would be giving us? To put it in reverse, our\nattitude being \"we trust the first copy of object we saw\", which\ntranslates to \"we trust where we explicitly clone and fetch from\" in\nthe traditional world without lazy fetching, if somebody else we are\nexplicitly fetching from offers us an object that the promisor\nremote would give us, we just do not bother if they are the same\nbecause it is not like we trust our promisor more than we trust the\ncurrent counterpart we are fetching from?\n\n> Signed-off-by: Patrick Steinhardt \n> ---\n> builtin/index-pack.c | 4 +---\n> 1 file changed, 1 insertion(+), 3 deletions(-)\n>\n> diff --git a/builtin/index-pack.c b/builtin/index-pack.c\n> index f49431d626b..805b7aa1e28 100644\n> --- a/builtin/index-pack.c\n> +++ b/builtin/index-pack.c\n> @@ -892,9 +892,7 @@ static void sha1_object(const void *data, struct object_entry *obj_entry,\n> \n> \tif (startup_info->have_repository) {\n> \t\tread_lock();\n> -\t\tcollision_test_needed =\n> -\t\t\trepo_has_object_file_with_flags(the_repository, oid,\n> -\t\t\t\t\t\t\tOBJECT_INFO_QUICK);\n> +\t\tcollision_test_needed = has_object(the_repository, oid, 0);\n> \t\tread_unlock();\n> \t}\n"},{"id":"516920","messageId":"xmqqy0vk9crx.fsf@gitster.g","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-8-63f1695b7700@pks.im","subject":"Re: [PATCH v2 08/13] builtin/show-ref: don't fetch objects when printing refs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-28T21:50:10Z","receivedAt":"2025-04-28T21:50:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> When printing references via git-show-ref(1) we double check that their\n> respective object IDs point to an existing object. This check is\n> performed via `repo_has_object_file()`, which knows to fetch missing\n> promised objects in the background. We shouldn't have a need to fetch\n> such objects though as no reference should ever point to a missing\n> object at all.\n\nBut hasn't the earlier part of this series made sure that asking\nhas_object() without an extra flag would make it say \"no\" if the\nobject that we know we can obtain is kept in our promisor remote\nto be lazily fetched? Or am I missing some other mechanism to make\nsure that somebody does any necessary lazy fetching of an object\nbefore we point our refs at it?\n\nThanks.\n"},{"id":"516921","messageId":"xmqqtt689cmk.fsf@gitster.g","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-9-63f1695b7700@pks.im","subject":"Re: [PATCH v2 09/13] refs: don't fetch promisor objects in `ref_resolves_to_object()`","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-28T21:53:23Z","receivedAt":"2025-04-28T21:53:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Similar to the preceding commit, don't try to fetch objects pointed to\n> by references. Any reference whose object does not exist is broken by\n> definition, so we should report it accordingly.\n\nIt has always been correct that any reference whose object does not\nexist is broken by definition.\n\nBut didn't \"does not exist\" use to mean \"not in this repository,\nand cannot be obtained from our promisor remotes\", but with this\nseries, its meaning has changed to mean \"not in this repository\nright now---it does not matter if our promisor has it\"?\n\nSo with this change, aren't we changing that statement to \"any\nreference whose object does not exist may be broken, but we do not\nknow it until we consult our promisor remotes, if any\"?\n\n> Signed-off-by: Patrick Steinhardt \n> ---\n> refs.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/refs.c b/refs.c\n> index 6559db37890..0492cf0d13a 100644\n> --- a/refs.c\n> +++ b/refs.c\n> @@ -376,7 +376,7 @@ int ref_resolves_to_object(const char *refname,\n> {\n> \tif (flags & REF_ISBROKEN)\n> \t\treturn 0;\n> -\tif (!repo_has_object_file(repo, oid)) {\n> +\tif (!has_object(repo, oid, HAS_OBJECT_RECHECK_PACKED)) {\n> \t\terror(_(\"%s does not point to a valid object!\"), refname);\n> \t\treturn 0;\n> \t}\n"},{"id":"516922","messageId":"xmqqmsc09ci6.fsf@gitster.g","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-10-63f1695b7700@pks.im","subject":"Re: [PATCH v2 10/13] http-walker: don't fetch objects via promisor remotes","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-28T21:56:01Z","receivedAt":"2025-04-28T21:56:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> The HTTP walker is responsible for fetching objects via the dumb HTTP\n> protocol. To avoid re-fetching objects that we already have locally we\n> first check whether the object already exists in the local repository\n> before issuing the requests.\n>\n> This existence check is done by calling `repo_has_object_file()`, which\n> will fetch the object via a promisor remote in case it is in a promisor\n> pack. This fetch does not make any sense for us though: we're already in\n> the process of fetching the object anyway, so fetching it via a separate\n> connection is wasteful, but should otherwise be harmless.\n\nThis rationale does make sense for this code path. It half matches\nthe rationale of omitting the collision check when you see an object\nin the pack stream being indexed that it can also be obtained from\nyour promisor remotes, which was of dubious validity, but in this\ncase it makes perfect sense.\n\nThanks.\n\n"},{"id":"516923","messageId":"xmqqikmo9bzc.fsf@gitster.g","threadId":"63335","inReplyTo":"20250425-pks-object-store-cleanups-v2-12-63f1695b7700@pks.im","subject":"Re: [PATCH v2 12/13] bulk-checkin: don't fetch promised objects on write","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-28T22:07:19Z","receivedAt":"2025-04-28T22:07:21Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> In an ideal world, we would protect against this by fetching the\n> promised object and then performing a collision check. But this feels\n> exceedingly expensive and ultimately rather pointless, as more common\n> writing paths like `write_loose_object()` don't protect against this\n> scenario either.\n\nWhen writing loose object, wouldn't collision check kick in, and\ndidn't we compare \"existing (not here but virtually here due to\npromisor)\" object and what write_loose_object() tried to create, at\nleast before this series which may (or may not; I lost track) have\ndisabled that check?\n\nI think the overall goal of deprecating the function with long name\nwith another function with a short-and-sweet name with different\ndefault is a worthy thing, and while I do agree with \"as we are\nreplacing function with another with different default, we need to\npass different flags to keep the same behaviour\" early parts of the\nseries, I am not sure about these latter steps.\n\nThanks.\n"},{"id":"516929","messageId":"aBBukgbWZRRLgnLl@pks.im","threadId":"63335","inReplyTo":"xmqqikmo9bzc.fsf@gitster.g","subject":"Re: [PATCH v2 12/13] bulk-checkin: don't fetch promised objects on write","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T06:15:46Z","receivedAt":"2025-04-29T06:15:54Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Apr 28, 2025 at 03:07:19PM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > In an ideal world, we would protect against this by fetching the\n> > promised object and then performing a collision check. But this feels\n> > exceedingly expensive and ultimately rather pointless, as more common\n> > writing paths like `write_loose_object()` don't protect against this\n> > scenario either.\n> \n> When writing loose object, wouldn't collision check kick in, and\n> didn't we compare \"existing (not here but virtually here due to\n> promisor)\" object and what write_loose_object() tried to create, at\n> least before this series which may (or may not; I lost track) have\n> disabled that check?\n> \n> I think the overall goal of deprecating the function with long name\n> with another function with a short-and-sweet name with different\n> default is a worthy thing, and while I do agree with \"as we are\n> replacing function with another with different default, we need to\n> pass different flags to keep the same behaviour\" early parts of the\n> series, I am not sure about these latter steps.\n\nYeah, to be honest I wasn't totally sure whether to include these steps\nmyself as I anticipated that they will lead to discussions that derail\nmy original goal, which is to clean up the interfaces in the object\nsubsystem. I decided to go with these where I thought that my train of\nthought is reasonable, but given your comments I'll probably just drop\nthose patches.\n\nWe can still adapt these callsites in the future as needed.\n\nPatrick\n"},{"id":"516930","messageId":"aBBumhDhWoR9LEb3@pks.im","threadId":"63335","inReplyTo":"xmqq34dsarhv.fsf@gitster.g","subject":"Re: [PATCH v2 07/13] builtin/index-pack: don't fetch promised objects for collision check","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T06:15:54Z","receivedAt":"2025-04-29T06:15:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Apr 28, 2025 at 02:46:52PM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> \n> > Any packed objects indexed via git-index-pack(1) are subject to a\n> > collision check. This collision check has the intent to determine\n> > whether we already have an object with the same object ID, but different\n> > contents in the repository.\n> >\n> > The check whether the collision check is really needed is only performed\n> > in case `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)` tells\n> > us that the object exists. But unless explicitly told otherwise by\n> > passing `OBJECT_INFO_SKIP_FETCH_OBJECT`, this function will also cause\n> > us to fetch the object in case it is part of a promisor pack. As such,\n> > we may end up fetching the object only to check whether the fetched\n> > object and the object that we're indexing have the same content.\n> >\n> > This behaviour is highly dubious and more likely than not unintended.\n> > Fix it by converting to `has_object()`, which knows to neither reload\n> > packfiles nor to fetch promisor objects by default.\n> \n> It is unclear why you thing it is highly dubious from reading the\n> above paragraph three times, though.\n> \n> Is it that if we are suspicious of the incoming pack data we are\n> indexing, we should also not be too trusting of the object that our\n> promisor remote would be giving us? To put it in reverse, our\n> attitude being \"we trust the first copy of object we saw\", which\n> translates to \"we trust where we explicitly clone and fetch from\" in\n> the traditional world without lazy fetching, if somebody else we are\n> explicitly fetching from offers us an object that the promisor\n> remote would give us, we just do not bother if they are the same\n> because it is not like we trust our promisor more than we trust the\n> current counterpart we are fetching from?\n\nYes, exactly. When we don't have an object locally we don't have a trust\nanchor for verifying that contents of the object look as expected. So\nthere are only two ways to do this:\n\n - Use a trust-on-first-use model. We trust the object we obtain\n initially and from thereon we start to treat it as the \"correct\"\n object and verify incoming objects with the same ID against it.\n\n - We only trust what everyone agrees one. In that case though we\n really should be cross-verifying with _all_ remotes, not only with\n the promisor remote.\n\nRight now we do neither, but we end up treating the promisor as \"more\ntrusted\" than any of the other remotes.\n\nI think it's completely unintentional that we end up fetching the object\nfrom the promisor to perform a collision check against the packfile we\nare about to index. It is highly likely that the promisor remote and the\nremote that we're fetching from are the same anyway, so all this does is\nto waste resources.\n\nAnyway, I'll evict these patches from this series. I think a couple of\nthe sites are broken, but for now I care more about the bigger picture.\n\nPatrick\n"},{"id":"516933","messageId":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","threadId":"63335","inReplyTo":"20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im","subject":"[PATCH v3 0/7] object-store: a handful of cleanups","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T07:52:14Z","receivedAt":"2025-04-29T07:52:24Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series contains a handful of cleanups to the object store\nsubsystem:\n\n - A couple of definitions are moved out of \"object-store.h\" as they\n belong to other subsystems.\n\n - Some functions are dropped and/or renamed.\n\n - The biggest part is the removal of `repo_has_object_file()`. This\n function and its `_with_flags()` variant are marked as deprecated,\n with the replacement being `has_object()`. The benefit of that\n function is that it doesn't reload packfiles and doesn't fetch\n promisor objects by default so that it becomes more explicit when\n one really wants to do so.\n\nThese cleanups are in preparation for getting rid of `the_repository` in\n\"object-store.c\".\n\nThe patch series is built on top of 4bbb303af69 (The seventh batch,\n2025-04-17) with ps/object-file-cleanup at 68cd492a3e6 (object-store:\nmerge \"object-store-ll.h\" and \"object-store.h\", 2025-04-15) merged into\nit.\n\nChanges in v2:\n - A handful of improvements for commit messages.\n - Link to v1: https://lore.kernel.org/r/20250423-pks-object-store-cleanups-v1-0-81f8411a5d08@pks.im\n\nChanges in v3:\n - Move around `hash_object_file()`, which I missed in previous\n iterations.\n - Don't try to fix up callsites where we end up fetching promised\n objects. This patch series now only does a trivial 1:1 conversion.\n - Clarify why we're sunsetting `repo_has_object_file()`.\n - Link to v2: https://lore.kernel.org/r/20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (7):\n object-store: move `struct packed_git` into \"packfile.h\"\n object-store: drop `loose_object_path()`\n object-store: move and rename `odb_pack_keep()`\n object-store: move function declarations to their respective subsystems\n object-store: allow fetching objects via `has_object()`\n treewide: convert users of `repo_has_object_file()` to `has_object()`\n object-store: drop `repo_has_object_file()`\n\n builtin/cat-file.c | 3 +-\n builtin/clone.c | 4 +-\n builtin/count-objects.c | 2 +-\n builtin/fast-import.c | 3 +-\n builtin/fetch.c | 15 ++--\n builtin/gc.c | 2 +-\n builtin/index-pack.c | 7 +-\n builtin/receive-pack.c | 4 +-\n builtin/remote.c | 3 +-\n builtin/show-ref.c | 3 +-\n builtin/unpack-objects.c | 3 +-\n bulk-checkin.c | 3 +-\n cache-tree.c | 13 +++-\n convert.c | 2 +-\n diffcore-rename.c | 2 +-\n dir.c | 2 +-\n fetch-pack.c | 7 +-\n http-push.c | 11 ++-\n http-walker.c | 9 ++-\n http.c | 4 +-\n list-objects.c | 3 +-\n log-tree.c | 2 +-\n notes.c | 3 +-\n object-file.c | 4 +-\n object-file.h | 81 ++++++++++++++++++++\n object-name.c | 2 +-\n object-store.c | 44 ++---------\n object-store.h | 195 +++--------------------------------------------\n pack-objects.h | 1 +\n packfile.h | 78 ++++++++++++++++++-\n path.c | 14 ++++\n path.h | 7 ++\n prune-packed.c | 2 +-\n reachable.c | 2 +-\n reflog.c | 3 +-\n refs.c | 2 +-\n remote.c | 2 +-\n send-pack.c | 5 +-\n shallow.c | 9 ++-\n upload-pack.c | 3 +-\n walker.c | 3 +-\n 41 files changed, 278 insertions(+), 289 deletions(-)\n\nRange-diff versus v2:\n\n 1: 3b0282e9621 = 1: ade4c11cb5c object-store: move `struct packed_git` into \"packfile.h\"\n 2: 60871520f14 = 2: a38ae18bf91 object-store: drop `loose_object_path()`\n 3: a3eb7be5cc4 = 3: cb14eafdffa object-store: move and rename `odb_pack_keep()`\n 4: 2fbf97b6826 ! 4: d1b3c0a23f9 object-store: move function declarations to their respective subsystems\n @@ builtin/gc.c\n #include \"pack-objects.h\"\n #include \"path.h\"\n \n + ## convert.c ##\n +@@\n + #include \"copy.h\"\n + #include \"gettext.h\"\n + #include \"hex.h\"\n +-#include \"object-store.h\"\n ++#include \"object-file.h\"\n + #include \"attr.h\"\n + #include \"run-command.h\"\n + #include \"quote.h\"\n +\n + ## diffcore-rename.c ##\n +@@\n + #include \"git-compat-util.h\"\n + #include \"diff.h\"\n + #include \"diffcore.h\"\n +-#include \"object-store.h\"\n ++#include \"object-file.h\"\n + #include \"hashmap.h\"\n + #include \"mem-pool.h\"\n + #include \"oid-array.h\"\n +\n + ## dir.c ##\n +@@\n + #include \"environment.h\"\n + #include \"gettext.h\"\n + #include \"name-hash.h\"\n +-#include \"object-store.h\"\n ++#include \"object-file.h\"\n + #include \"path.h\"\n + #include \"refs.h\"\n + #include \"repository.h\"\n +\n + ## log-tree.c ##\n +@@\n + #include \"environment.h\"\n + #include \"hex.h\"\n + #include \"object-name.h\"\n +-#include \"object-store.h\"\n ++#include \"object-file.h\"\n + #include \"repository.h\"\n + #include \"tmp-objdir.h\"\n + #include \"commit.h\"\n +\n ## object-file.h ##\n @@\n \n @@ object-file.h: int has_loose_object_nonlocal(const struct object_id *);\n /**\n * format_object_header() is a thin wrapper around s xsnprintf() that\n * writes the initial \" \" part of the loose object\n +@@ object-file.h: int finalize_object_file(const char *tmpfile, const char *filename);\n + int finalize_object_file_flags(const char *tmpfile, const char *filename,\n + \t\t\t enum finalize_object_file_flags flags);\n + \n ++void hash_object_file(const struct git_hash_algo *algo, const void *buf,\n ++\t\t unsigned long len, enum object_type type,\n ++\t\t struct object_id *oid);\n ++\n + /* Helper to check and \"touch\" a file */\n + int check_and_freshen_file(const char *fn, int freshen);\n + \n \n ## object-name.c ##\n @@\n @@ object-store.h: void raw_object_store_clear(struct raw_object_store *o);\n void *repo_read_object_file(struct repository *r,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type *type,\n +@@ object-store.h: void *repo_read_object_file(struct repository *r,\n + /* Read and unpack an object file into memory, write memory to an object file */\n + int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);\n + \n +-void hash_object_file(const struct git_hash_algo *algo, const void *buf,\n +-\t\t unsigned long len, enum object_type type,\n +-\t\t struct object_id *oid);\n +-\n + /*\n + * Add an object file to the in-memory object store, without writing it\n + * to disk.\n @@ object-store.h: static inline void obj_read_unlock(void)\n \tif(obj_read_use_lock)\n \t\tpthread_mutex_unlock(&obj_read_mutex);\n 5: 6e165b497e4 ! 5: 7b8a6286f6a object-store: allow fetching objects via `has_object()`\n @@ Commit message\n object-store: allow fetching objects via `has_object()`\n \n We're about to fully remove `repo_has_object_file()` in favor of\n - `has_object()` given that the latter has better defaults: it neither\n - reloads packfiles by default nor does it fetch any promised objects in\n - case they are missing.\n -\n - The latter usecase keeps us from converting a couple of callsites that\n - currently do fetch objects though. It is not really clear whether _all_\n - of those callsites should be fetching objects, but for a subset of them\n - it is the desired behaviour indeed.\n + `has_object()`. The latter function does not yet have a way to fetch\n + missing objects via a promisor remote though, which means that it cannot\n + fully replace all usecases of `repo_has_object_file()`.\n \n Introduce a new flag `HAS_OBJECT_FETCH_PROMISOR` that causes the\n function to optionally fetch missing objects which are part of a\n - promisor pack.\n + promisor pack. This flag will be used in the subsequent commit.\n \n Signed-off-by: Patrick Steinhardt \n \n 6: 325178adf42 ! 6: a7733b66fd6 treewide: trivial conversions of `repo_has_object_file()`\n @@ Metadata\n Author: Patrick Steinhardt \n \n ## Commit message ##\n - treewide: trivial conversions of `repo_has_object_file()`\n + treewide: convert users of `repo_has_object_file()` to `has_object()`\n \n As the comment of `repo_has_object_file()` and its `_with_flags()`\n variant tells us, these functions are considered to be deprecated in\n - favor of `has_object()`. The benefit of the replacement is better\n - defaults: it doesn't fetch missing objects via promisor remotes, and\n - neither does it reload packfiles if an object wasn't found by default.\n + favor of `has_object()`. There are a couple of slight benefits in favor\n + of the replacement:\n \n - Start sunsetting the functions by replacing trivial callsites of it with\n - `has_object()`:\n + - The new function has a short-and-sweet name.\n +\n + - More explicit defaults: `has_object()` doesn't fetch missing objects\n + via promisor remotes, and neither does it reload packfiles if an\n + object wasn't found by default. This ensures that it becomes\n + immediately obvious when a simple object existence check may result\n + in expensive actions.\n +\n + Most importantly though, it is confusing that we have two sets of\n + functions that ultimately do the same thing, but with different\n + defaults.\n +\n + Start sunsetting `repo_has_object_file()` and its `_with_flags()`\n + sibling by replacing all callsites with `has_object()`:\n \n - `repo_has_object_file(...)` is equivalent to\n `has_object(..., HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)`.\n @@ Commit message\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_SKIP_FETCH_OBJECT)`\n is equivalent to `has_object(..., HAS_OBJECT_RECHECK_PACKED)`.\n \n - While converting callsites to use `HAS_OBJECT_FETCH_PROMISOR` retains\n - the current behaviour, it is not entirely clear whether all of them\n - really should be fetching promised objects. A subset of such callsites\n - where it is obvious that they really shouldn't fetch objects has been\n - left out of this commit and will be adapted over the next couple of\n - commits.\n + - `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)`\n + is equivalent to `has_object(..., HAS_OBJECT_FETCH_PROMISOR)`.\n +\n + The replacements should be functionally equivalent.\n \n Signed-off-by: Patrick Steinhardt \n \n @@ builtin/fetch.c: static int check_exist_and_connected(struct ref *ref_map)\n \t}\n \n \n + ## builtin/index-pack.c ##\n +@@ builtin/index-pack.c: static void sha1_object(const void *data, struct object_entry *obj_entry,\n + \n + \tif (startup_info->have_repository) {\n + \t\tread_lock();\n +-\t\tcollision_test_needed =\n +-\t\t\trepo_has_object_file_with_flags(the_repository, oid,\n +-\t\t\t\t\t\t\tOBJECT_INFO_QUICK);\n ++\t\tcollision_test_needed = has_object(the_repository, oid,\n ++\t\t\t\t\t\t HAS_OBJECT_FETCH_PROMISOR);\n + \t\tread_unlock();\n + \t}\n + \n +\n ## builtin/receive-pack.c ##\n @@ builtin/receive-pack.c: static const char *update(struct command *cmd, struct shallow_info *si)\n \t\t}\n @@ builtin/remote.c: static int get_push_ref_states(const struct ref *remote_refs,\n \t\t\tinfo->status = PUSH_STATUS_FASTFORWARD;\n \t\telse\n \n + ## builtin/show-ref.c ##\n +@@ builtin/show-ref.c: static void show_one(const struct show_one_options *opts,\n + \tconst char *hex;\n + \tstruct object_id peeled;\n + \n +-\tif (!repo_has_object_file(the_repository, oid))\n ++\tif (!has_object(the_repository, oid,\n ++\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n + \t\tdie(\"git show-ref: bad ref %s (%s)\", refname,\n + \t\t oid_to_hex(oid));\n + \n +\n ## builtin/unpack-objects.c ##\n @@ builtin/unpack-objects.c: static void unpack_delta_entry(enum object_type type, unsigned long delta_size,\n \t\tdelta_data = get_data(delta_size);\n @@ builtin/unpack-objects.c: static void unpack_delta_entry(enum object_type type,\n \t\telse if (resolve_against_held(nr, &base_oid,\n \t\t\t\t\t delta_data, delta_size))\n \n + ## bulk-checkin.c ##\n +@@ bulk-checkin.c: static void flush_batch_fsync(void)\n + static int already_written(struct bulk_checkin_packfile *state, struct object_id *oid)\n + {\n + \t/* The object may already exist in the repository */\n +-\tif (repo_has_object_file(the_repository, oid))\n ++\tif (has_object(the_repository, oid,\n ++\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n + \t\treturn 1;\n + \n + \t/* Might want to keep the list sorted */\n +\n ## cache-tree.c ##\n @@ cache-tree.c: int cache_tree_fully_valid(struct cache_tree *it)\n \tint i;\n @@ http-push.c: int cmd_main(int argc, const char **argv)\n \t\t\t\t &ref->old_oid)) {\n \t\t\t\t/*\n \n + ## http-walker.c ##\n +@@ http-walker.c: static int fill_active_slot(void *data UNUSED)\n + \tlist_for_each_safe(pos, tmp, head) {\n + \t\tobj_req = list_entry(pos, struct object_request, node);\n + \t\tif (obj_req->state == WAITING) {\n +-\t\t\tif (repo_has_object_file(the_repository, &obj_req->oid))\n ++\t\t\tif (has_object(the_repository, &obj_req->oid,\n ++\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n + \t\t\t\tobj_req->state = COMPLETE;\n + \t\t\telse {\n + \t\t\t\tstart_object_request(obj_req);\n +@@ http-walker.c: static int fetch_object(struct walker *walker, const struct object_id *oid)\n + \tif (!obj_req)\n + \t\treturn error(\"Couldn't find request for %s in the queue\", hex);\n + \n +-\tif (repo_has_object_file(the_repository, &obj_req->oid)) {\n ++\tif (has_object(the_repository, &obj_req->oid,\n ++\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n + \t\tif (obj_req->req)\n + \t\t\tabort_http_object_request(&obj_req->req);\n + \t\tabort_object_request(obj_req);\n +\n + ## list-objects.c ##\n +@@ list-objects.c: static void process_blob(struct traversal_context *ctx,\n + \t * of missing objects.\n + \t */\n + \tif (ctx->revs->exclude_promisor_objects &&\n +-\t !repo_has_object_file(the_repository, &obj->oid) &&\n ++\t !has_object(the_repository, &obj->oid,\n ++\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) &&\n + \t is_promisor_object(ctx->revs->repo, &obj->oid))\n + \t\treturn;\n + \n +\n ## notes.c ##\n @@ notes.c: static int prune_notes_helper(const struct object_id *object_oid,\n \tstruct note_delete_list **l = (struct note_delete_list **) cb_data;\n @@ reflog.c: static int tree_is_complete(const struct object_id *oid)\n \t\t\ttree->object.flags |= INCOMPLETE;\n \t\t\tcomplete = 0;\n \n + ## refs.c ##\n +@@ refs.c: int ref_resolves_to_object(const char *refname,\n + {\n + \tif (flags & REF_ISBROKEN)\n + \t\treturn 0;\n +-\tif (!repo_has_object_file(repo, oid)) {\n ++\tif (!has_object(repo, oid, HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n + \t\terror(_(\"%s does not point to a valid object!\"), refname);\n + \t\treturn 0;\n + \t}\n +\n ## remote.c ##\n @@ remote.c: void set_ref_status_for_push(struct ref *remote_refs, int send_mirror,\n \t\tif (!reject_reason && !ref->deletion && !is_null_oid(&ref->old_oid)) {\n 7: ab0dc1c5ce3 < -: ----------- builtin/index-pack: don't fetch promised objects for collision check\n 8: 89c314a7dfd < -: ----------- builtin/show-ref: don't fetch objects when printing refs\n 9: a1f647f264f < -: ----------- refs: don't fetch promisor objects in `ref_resolves_to_object()`\n10: 624aa140720 < -: ----------- http-walker: don't fetch objects via promisor remotes\n11: a2604942569 < -: ----------- list-objects: clarify how promised blobs are excluded\n12: d08d6092580 < -: ----------- bulk-checkin: don't fetch promised objects on write\n13: 84af5509257 = 7: 60aeb060bd5 object-store: drop `repo_has_object_file()`\n\n---\nbase-commit: ca819c0751cedd1713334882e4c83687f8478a54\nchange-id: 20250422-pks-object-store-cleanups-5a6077014155\n\n"},{"id":"516934","messageId":"20250429-pks-object-store-cleanups-v3-1-42902bad09a1@pks.im","threadId":"63335","inReplyTo":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","subject":"[PATCH v3 1/7] object-store: move `struct packed_git` into \"packfile.h\"","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T07:52:15Z","receivedAt":"2025-04-29T07:52:25Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The \"object-store.h\" header contains the definition of `struct\npacked_git`. As this structure hosts all kind of information about a\nspecific packfile it is arguably a bit out of place in a generic place\nlike \"object-store.h\".\n\nMove the structure as well as `pack_map_entry_cmp()` into \"packfile.h\".\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.h | 59 +---------------------------------------------------------\n pack-objects.h | 1 +\n packfile.h | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-\n 3 files changed, 60 insertions(+), 59 deletions(-)\n\ndiff --git a/object-store.h b/object-store.h\nindex 46961dc9542..e04469a85fb 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -92,65 +92,8 @@ struct oidtree *odb_loose_cache(struct object_directory *odb,\n /* Empty the loose object cache for the specified object directory. */\n void odb_clear_loose_cache(struct object_directory *odb);\n \n-struct packed_git {\n-\tstruct hashmap_entry packmap_ent;\n-\tstruct packed_git *next;\n-\tstruct list_head mru;\n-\tstruct pack_window *windows;\n-\toff_t pack_size;\n-\tconst void *index_data;\n-\tsize_t index_size;\n-\tuint32_t num_objects;\n-\tsize_t crc_offset;\n-\tstruct oidset bad_objects;\n-\tint index_version;\n-\ttime_t mtime;\n-\tint pack_fd;\n-\tint index; /* for builtin/pack-objects.c */\n-\tunsigned pack_local:1,\n-\t\t pack_keep:1,\n-\t\t pack_keep_in_core:1,\n-\t\t freshened:1,\n-\t\t do_not_close:1,\n-\t\t pack_promisor:1,\n-\t\t multi_pack_index:1,\n-\t\t is_cruft:1;\n-\tunsigned char hash[GIT_MAX_RAWSZ];\n-\tstruct revindex_entry *revindex;\n-\tconst uint32_t *revindex_data;\n-\tconst uint32_t *revindex_map;\n-\tsize_t revindex_size;\n-\t/*\n-\t * mtimes_map points at the beginning of the memory mapped region of\n-\t * this pack's corresponding .mtimes file, and mtimes_size is the size\n-\t * of that .mtimes file\n-\t */\n-\tconst uint32_t *mtimes_map;\n-\tsize_t mtimes_size;\n-\n-\t/* repo denotes the repository this packfile belongs to */\n-\tstruct repository *repo;\n-\n-\t/* something like \".git/objects/pack/xxxxx.pack\" */\n-\tchar pack_name[FLEX_ARRAY]; /* more */\n-};\n-\n+struct packed_git;\n struct multi_pack_index;\n-\n-static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,\n-\t\t\t\t const struct hashmap_entry *entry,\n-\t\t\t\t const struct hashmap_entry *entry2,\n-\t\t\t\t const void *keydata)\n-{\n-\tconst char *key = keydata;\n-\tconst struct packed_git *pg1, *pg2;\n-\n-\tpg1 = container_of(entry, const struct packed_git, packmap_ent);\n-\tpg2 = container_of(entry2, const struct packed_git, packmap_ent);\n-\n-\treturn strcmp(pg1->pack_name, key ? key : pg2->pack_name);\n-}\n-\n struct cached_object_entry;\n \n struct raw_object_store {\ndiff --git a/pack-objects.h b/pack-objects.h\nindex d1c4ae7f9b6..475a2d67ce3 100644\n--- a/pack-objects.h\n+++ b/pack-objects.h\n@@ -4,6 +4,7 @@\n #include \"object-store.h\"\n #include \"thread-utils.h\"\n #include \"pack.h\"\n+#include \"packfile.h\"\n \n struct repository;\n \ndiff --git a/packfile.h b/packfile.h\nindex 25097213d06..05499382397 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -1,13 +1,70 @@\n #ifndef PACKFILE_H\n #define PACKFILE_H\n \n+#include \"list.h\"\n #include \"object.h\"\n #include \"oidset.h\"\n \n /* in object-store.h */\n-struct packed_git;\n struct object_info;\n \n+struct packed_git {\n+\tstruct hashmap_entry packmap_ent;\n+\tstruct packed_git *next;\n+\tstruct list_head mru;\n+\tstruct pack_window *windows;\n+\toff_t pack_size;\n+\tconst void *index_data;\n+\tsize_t index_size;\n+\tuint32_t num_objects;\n+\tsize_t crc_offset;\n+\tstruct oidset bad_objects;\n+\tint index_version;\n+\ttime_t mtime;\n+\tint pack_fd;\n+\tint index; /* for builtin/pack-objects.c */\n+\tunsigned pack_local:1,\n+\t\t pack_keep:1,\n+\t\t pack_keep_in_core:1,\n+\t\t freshened:1,\n+\t\t do_not_close:1,\n+\t\t pack_promisor:1,\n+\t\t multi_pack_index:1,\n+\t\t is_cruft:1;\n+\tunsigned char hash[GIT_MAX_RAWSZ];\n+\tstruct revindex_entry *revindex;\n+\tconst uint32_t *revindex_data;\n+\tconst uint32_t *revindex_map;\n+\tsize_t revindex_size;\n+\t/*\n+\t * mtimes_map points at the beginning of the memory mapped region of\n+\t * this pack's corresponding .mtimes file, and mtimes_size is the size\n+\t * of that .mtimes file\n+\t */\n+\tconst uint32_t *mtimes_map;\n+\tsize_t mtimes_size;\n+\n+\t/* repo denotes the repository this packfile belongs to */\n+\tstruct repository *repo;\n+\n+\t/* something like \".git/objects/pack/xxxxx.pack\" */\n+\tchar pack_name[FLEX_ARRAY]; /* more */\n+};\n+\n+static inline int pack_map_entry_cmp(const void *cmp_data UNUSED,\n+\t\t\t\t const struct hashmap_entry *entry,\n+\t\t\t\t const struct hashmap_entry *entry2,\n+\t\t\t\t const void *keydata)\n+{\n+\tconst char *key = keydata;\n+\tconst struct packed_git *pg1, *pg2;\n+\n+\tpg1 = container_of(entry, const struct packed_git, packmap_ent);\n+\tpg2 = container_of(entry2, const struct packed_git, packmap_ent);\n+\n+\treturn strcmp(pg1->pack_name, key ? key : pg2->pack_name);\n+}\n+\n struct pack_window {\n \tstruct pack_window *next;\n \tunsigned char *base;\n\n-- \n2.49.0.967.g6a0df3ecc3.dirty\n\n"},{"id":"516935","messageId":"20250429-pks-object-store-cleanups-v3-2-42902bad09a1@pks.im","threadId":"63335","inReplyTo":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","subject":"[PATCH v3 2/7] object-store: drop `loose_object_path()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T07:52:16Z","receivedAt":"2025-04-29T07:52:26Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The function `loose_object_path()` is a trivial wrapper around\n`odb_loose_path()`, with the only exception that it always uses the\nprimary object database of the given repository. This doesn't really add\na ton of value though, so let's drop the function and inline it at every\ncallsite.\n\nSigned-off-by: Patrick Steinhardt \n---\n http-walker.c | 3 ++-\n http.c | 4 ++--\n object-file.c | 4 ++--\n object-file.h | 4 ++++\n object-store.c | 6 ------\n object-store.h | 7 -------\n 6 files changed, 10 insertions(+), 18 deletions(-)\n\ndiff --git a/http-walker.c b/http-walker.c\nindex 882cae19c24..95458e2f638 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -9,6 +9,7 @@\n #include \"list.h\"\n #include \"transport.h\"\n #include \"packfile.h\"\n+#include \"object-file.h\"\n #include \"object-store.h\"\n \n struct alt_base {\n@@ -540,7 +541,7 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)\n \t\tret = error(\"File %s has bad hash\", hex);\n \t} else if (req->rename < 0) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tloose_object_path(the_repository, &buf, &req->oid);\n+\t\todb_loose_path(the_repository->objects->odb, &buf, &req->oid);\n \t\tret = error(\"unable to write sha1 filename %s\", buf.buf);\n \t\tstrbuf_release(&buf);\n \t}\ndiff --git a/http.c b/http.c\nindex 0c411380425..3c029cf8947 100644\n--- a/http.c\n+++ b/http.c\n@@ -2662,7 +2662,7 @@ struct http_object_request *new_http_object_request(const char *base_url,\n \toidcpy(&freq->oid, oid);\n \tfreq->localfile = -1;\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \tstrbuf_addf(&freq->tmpfile, \"%s.temp\", filename.buf);\n \n \tstrbuf_addf(&prevfile, \"%s.prev\", filename.buf);\n@@ -2814,7 +2814,7 @@ int finish_http_object_request(struct http_object_request *freq)\n \t\tunlink_or_warn(freq->tmpfile.buf);\n \t\treturn -1;\n \t}\n-\tloose_object_path(the_repository, &filename, &freq->oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, &freq->oid);\n \tfreq->rename = finalize_object_file(freq->tmpfile.buf, filename.buf);\n \tstrbuf_release(&filename);\n \ndiff --git a/object-file.c b/object-file.c\nindex 9cc3a24a40d..dc56a4766df 100644\n--- a/object-file.c\n+++ b/object-file.c\n@@ -932,7 +932,7 @@ static int write_loose_object(const struct object_id *oid, char *hdr,\n \tif (batch_fsync_enabled(FSYNC_COMPONENT_LOOSE_OBJECT))\n \t\tprepare_loose_object_bulk_checkin();\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \n \tfd = start_loose_object_common(&tmp_file, filename.buf, flags,\n \t\t\t\t &stream, compressed, sizeof(compressed),\n@@ -1079,7 +1079,7 @@ int stream_loose_object(struct input_stream *in_stream, size_t len,\n \t\tgoto cleanup;\n \t}\n \n-\tloose_object_path(the_repository, &filename, oid);\n+\todb_loose_path(the_repository->objects->odb, &filename, oid);\n \n \t/* We finally know the object path, and create the missing dir. */\n \tdirlen = directory_size(filename.buf);\ndiff --git a/object-file.h b/object-file.h\nindex c002fbe2345..0a7b6b9f9d9 100644\n--- a/object-file.h\n+++ b/object-file.h\n@@ -25,6 +25,10 @@ int index_path(struct index_state *istate, struct object_id *oid, const char *pa\n \n struct object_directory;\n \n+/*\n+ * Put in `buf` the name of the file in the local object database that\n+ * would be used to store a loose object with the specified oid.\n+ */\n const char *odb_loose_path(struct object_directory *odb,\n \t\t\t struct strbuf *buf,\n \t\t\t const struct object_id *oid);\ndiff --git a/object-store.c b/object-store.c\nindex 6ab50d25d3e..e5cfb8c0079 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -96,12 +96,6 @@ int odb_pack_keep(const char *name)\n \treturn open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n }\n \n-const char *loose_object_path(struct repository *r, struct strbuf *buf,\n-\t\t\t const struct object_id *oid)\n-{\n-\treturn odb_loose_path(r->objects->odb, buf, oid);\n-}\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\ndiff --git a/object-store.h b/object-store.h\nindex e04469a85fb..5668de62d01 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -196,13 +196,6 @@ int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n */\n int odb_pack_keep(const char *name);\n \n-/*\n- * Put in `buf` the name of the file in the local object database that\n- * would be used to store a loose object with the specified oid.\n- */\n-const char *loose_object_path(struct repository *r, struct strbuf *buf,\n-\t\t\t const struct object_id *oid);\n-\n void *map_loose_object(struct repository *r, const struct object_id *oid,\n \t\t unsigned long *size);\n \n\n-- \n2.49.0.967.g6a0df3ecc3.dirty\n\n"},{"id":"516936","messageId":"20250429-pks-object-store-cleanups-v3-3-42902bad09a1@pks.im","threadId":"63335","inReplyTo":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","subject":"[PATCH v3 3/7] object-store: move and rename `odb_pack_keep()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T07:52:17Z","receivedAt":"2025-04-29T07:52:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The function `odb_pack_keep()` creates a file at the passed-in path. If\nthis fails, then the function re-tries by first creating any potentially\nmissing leading directories and then trying to create the file once\nagain. As such, this function doesn't host any kind of logic that is\nspecific to the object store, but is rather a generic helper function.\n\nRename the function to `safe_create_file_with_leading_directories()` and\nmove it into \"path.c\". While at it, refactor it so that it loses its\ndependency on `the_repository`.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/fast-import.c | 3 ++-\n builtin/index-pack.c | 2 +-\n object-store.c | 13 -------------\n object-store.h | 7 -------\n path.c | 14 ++++++++++++++\n path.h | 7 +++++++\n 6 files changed, 24 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin/fast-import.c b/builtin/fast-import.c\nindex c1e198f4e34..b2839c5f439 100644\n--- a/builtin/fast-import.c\n+++ b/builtin/fast-import.c\n@@ -811,7 +811,8 @@ static char *keep_pack(const char *curr_index_name)\n \tint keep_fd;\n \n \todb_pack_name(pack_data->repo, &name, pack_data->hash, \"keep\");\n-\tkeep_fd = odb_pack_keep(name.buf);\n+\tkeep_fd = safe_create_file_with_leading_directories(pack_data->repo,\n+\t\t\t\t\t\t\t name.buf);\n \tif (keep_fd < 0)\n \t\tdie_errno(\"cannot create keep file\");\n \twrite_or_die(keep_fd, keep_msg, strlen(keep_msg));\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 60a8ee05dbc..f49431d626b 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -1565,7 +1565,7 @@ static void write_special_file(const char *suffix, const char *msg,\n \telse\n \t\tfilename = odb_pack_name(the_repository, &name_buf, hash, suffix);\n \n-\tfd = odb_pack_keep(filename);\n+\tfd = safe_create_file_with_leading_directories(the_repository, filename);\n \tif (fd < 0) {\n \t\tif (errno != EEXIST)\n \t\t\tdie_errno(_(\"cannot write %s file '%s'\"),\ndiff --git a/object-store.c b/object-store.c\nindex e5cfb8c0079..0cbad5a19a0 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -83,19 +83,6 @@ int odb_mkstemp(struct strbuf *temp_filename, const char *pattern)\n \treturn xmkstemp_mode(temp_filename->buf, mode);\n }\n \n-int odb_pack_keep(const char *name)\n-{\n-\tint fd;\n-\n-\tfd = open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n-\tif (0 <= fd)\n-\t\treturn fd;\n-\n-\t/* slow path */\n-\tsafe_create_leading_directories_const(the_repository, name);\n-\treturn open(name, O_RDWR|O_CREAT|O_EXCL, 0600);\n-}\n-\n /*\n * Return non-zero iff the path is usable as an alternate object database.\n */\ndiff --git a/object-store.h b/object-store.h\nindex 5668de62d01..aa8fc63043e 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -189,13 +189,6 @@ void raw_object_store_clear(struct raw_object_store *o);\n */\n int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n \n-/*\n- * Create a pack .keep file named \"name\" (which should generally be the output\n- * of odb_pack_name). Returns a file descriptor opened for writing, or -1 on\n- * error.\n- */\n-int odb_pack_keep(const char *name);\n-\n void *map_loose_object(struct repository *r, const struct object_id *oid,\n \t\t unsigned long *size);\n \ndiff --git a/path.c b/path.c\nindex 4505bb78e8b..3b598b2847f 100644\n--- a/path.c\n+++ b/path.c\n@@ -1011,6 +1011,20 @@ enum scld_error safe_create_leading_directories_const(struct repository *repo,\n \treturn result;\n }\n \n+int safe_create_file_with_leading_directories(struct repository *repo,\n+\t\t\t\t\t const char *path)\n+{\n+\tint fd;\n+\n+\tfd = open(path, O_RDWR|O_CREAT|O_EXCL, 0600);\n+\tif (0 <= fd)\n+\t\treturn fd;\n+\n+\t/* slow path */\n+\tsafe_create_leading_directories_const(repo, path);\n+\treturn open(path, O_RDWR|O_CREAT|O_EXCL, 0600);\n+}\n+\n static int have_same_root(const char *path1, const char *path2)\n {\n \tint is_abs1, is_abs2;\ndiff --git a/path.h b/path.h\nindex fd1a194b060..e67348f2539 100644\n--- a/path.h\n+++ b/path.h\n@@ -266,6 +266,13 @@ enum scld_error safe_create_leading_directories_const(struct repository *repo,\n \t\t\t\t\t\t const char *path);\n enum scld_error safe_create_leading_directories_no_share(char *path);\n \n+/*\n+ * Create a file, potentially creating its leading directories in case they\n+ * don't exist. Returns the return value of the open(3p) call.\n+ */\n+int safe_create_file_with_leading_directories(struct repository *repo,\n+\t\t\t\t\t const char *path);\n+\n # ifdef USE_THE_REPOSITORY_VARIABLE\n # include \"strbuf.h\"\n # include \"repository.h\"\n\n-- \n2.49.0.967.g6a0df3ecc3.dirty\n\n"},{"id":"516937","messageId":"20250429-pks-object-store-cleanups-v3-4-42902bad09a1@pks.im","threadId":"63335","inReplyTo":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","subject":"[PATCH v3 4/7] object-store: move function declarations to their respective subsystems","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T07:52:18Z","receivedAt":"2025-04-29T07:52:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"We carry declarations for a couple of functions in \"object-store.h\" that\nare not defined in \"object-store.c\", but in a different subsystem. Move\nthese declarations to the respective headers whose matching code files\ncarry the corresponding definition.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/count-objects.c | 2 +-\n builtin/gc.c | 2 +-\n convert.c | 2 +-\n diffcore-rename.c | 2 +-\n dir.c | 2 +-\n log-tree.c | 2 +-\n object-file.h | 77 +++++++++++++++++++++++++++++++++++++++\n object-name.c | 2 +-\n object-store.h | 95 +------------------------------------------------\n packfile.h | 19 ++++++++++\n prune-packed.c | 2 +-\n reachable.c | 2 +-\n 12 files changed, 106 insertions(+), 103 deletions(-)\n\ndiff --git a/builtin/count-objects.c b/builtin/count-objects.c\nindex 0bb5360b2f2..a88c0c9c09a 100644\n--- a/builtin/count-objects.c\n+++ b/builtin/count-objects.c\n@@ -12,7 +12,7 @@\n #include \"parse-options.h\"\n #include \"quote.h\"\n #include \"packfile.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n \n static unsigned long garbage;\n static off_t size_garbage;\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex b5ce1d32766..4d428f3253d 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -28,7 +28,7 @@\n #include \"commit.h\"\n #include \"commit-graph.h\"\n #include \"packfile.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"pack.h\"\n #include \"pack-objects.h\"\n #include \"path.h\"\ndiff --git a/convert.c b/convert.c\nindex 8783e17941f..b5f7cf6306c 100644\n--- a/convert.c\n+++ b/convert.c\n@@ -8,7 +8,7 @@\n #include \"copy.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"attr.h\"\n #include \"run-command.h\"\n #include \"quote.h\"\ndiff --git a/diffcore-rename.c b/diffcore-rename.c\nindex 179731462b5..7723bc3334e 100644\n--- a/diffcore-rename.c\n+++ b/diffcore-rename.c\n@@ -8,7 +8,7 @@\n #include \"git-compat-util.h\"\n #include \"diff.h\"\n #include \"diffcore.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"hashmap.h\"\n #include \"mem-pool.h\"\n #include \"oid-array.h\"\ndiff --git a/dir.c b/dir.c\nindex 5c4675b4ac4..e11342e1366 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -17,7 +17,7 @@\n #include \"environment.h\"\n #include \"gettext.h\"\n #include \"name-hash.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"path.h\"\n #include \"refs.h\"\n #include \"repository.h\"\ndiff --git a/log-tree.c b/log-tree.c\nindex a4d4ab59ca0..1d05dc1c701 100644\n--- a/log-tree.c\n+++ b/log-tree.c\n@@ -9,7 +9,7 @@\n #include \"environment.h\"\n #include \"hex.h\"\n #include \"object-name.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"repository.h\"\n #include \"tmp-objdir.h\"\n #include \"commit.h\"\ndiff --git a/object-file.h b/object-file.h\nindex 0a7b6b9f9d9..a85b2e5b494 100644\n--- a/object-file.h\n+++ b/object-file.h\n@@ -3,6 +3,7 @@\n \n #include \"git-zlib.h\"\n #include \"object.h\"\n+#include \"object-store.h\"\n \n struct index_state;\n \n@@ -25,6 +26,16 @@ int index_path(struct index_state *istate, struct object_id *oid, const char *pa\n \n struct object_directory;\n \n+/*\n+ * Populate and return the loose object cache array corresponding to the\n+ * given object ID.\n+ */\n+struct oidtree *odb_loose_cache(struct object_directory *odb,\n+\t\t\t\tconst struct object_id *oid);\n+\n+/* Empty the loose object cache for the specified object directory. */\n+void odb_clear_loose_cache(struct object_directory *odb);\n+\n /*\n * Put in `buf` the name of the file in the local object database that\n * would be used to store a loose object with the specified oid.\n@@ -42,6 +53,68 @@ int has_loose_object_nonlocal(const struct object_id *);\n \n int has_loose_object(const struct object_id *);\n \n+void *map_loose_object(struct repository *r, const struct object_id *oid,\n+\t\t unsigned long *size);\n+\n+/*\n+ * Iterate over the files in the loose-object parts of the object\n+ * directory \"path\", triggering the following callbacks:\n+ *\n+ * - loose_object is called for each loose object we find.\n+ *\n+ * - loose_cruft is called for any files that do not appear to be\n+ * loose objects. Note that we only look in the loose object\n+ * directories \"objects/[0-9a-f]{2}/\", so we will not report\n+ * \"objects/foobar\" as cruft.\n+ *\n+ * - loose_subdir is called for each top-level hashed subdirectory\n+ * of the object directory (e.g., \"$OBJDIR/f0\"). It is called\n+ * after the objects in the directory are processed.\n+ *\n+ * Any callback that is NULL will be ignored. Callbacks returning non-zero\n+ * will end the iteration.\n+ *\n+ * In the \"buf\" variant, \"path\" is a strbuf which will also be used as a\n+ * scratch buffer, but restored to its original contents before\n+ * the function returns.\n+ */\n+typedef int each_loose_object_fn(const struct object_id *oid,\n+\t\t\t\t const char *path,\n+\t\t\t\t void *data);\n+typedef int each_loose_cruft_fn(const char *basename,\n+\t\t\t\tconst char *path,\n+\t\t\t\tvoid *data);\n+typedef int each_loose_subdir_fn(unsigned int nr,\n+\t\t\t\t const char *path,\n+\t\t\t\t void *data);\n+int for_each_file_in_obj_subdir(unsigned int subdir_nr,\n+\t\t\t\tstruct strbuf *path,\n+\t\t\t\teach_loose_object_fn obj_cb,\n+\t\t\t\teach_loose_cruft_fn cruft_cb,\n+\t\t\t\teach_loose_subdir_fn subdir_cb,\n+\t\t\t\tvoid *data);\n+int for_each_loose_file_in_objdir(const char *path,\n+\t\t\t\t each_loose_object_fn obj_cb,\n+\t\t\t\t each_loose_cruft_fn cruft_cb,\n+\t\t\t\t each_loose_subdir_fn subdir_cb,\n+\t\t\t\t void *data);\n+int for_each_loose_file_in_objdir_buf(struct strbuf *path,\n+\t\t\t\t each_loose_object_fn obj_cb,\n+\t\t\t\t each_loose_cruft_fn cruft_cb,\n+\t\t\t\t each_loose_subdir_fn subdir_cb,\n+\t\t\t\t void *data);\n+\n+/*\n+ * Iterate over all accessible loose objects without respect to\n+ * reachability. By default, this includes both local and alternate objects.\n+ * The order in which objects are visited is unspecified.\n+ *\n+ * Any flags specific to packs are ignored.\n+ */\n+int for_each_loose_object(each_loose_object_fn, void *,\n+\t\t\t enum for_each_object_flags flags);\n+\n+\n /**\n * format_object_header() is a thin wrapper around s xsnprintf() that\n * writes the initial \" \" part of the loose object\n@@ -158,6 +231,10 @@ int finalize_object_file(const char *tmpfile, const char *filename);\n int finalize_object_file_flags(const char *tmpfile, const char *filename,\n \t\t\t enum finalize_object_file_flags flags);\n \n+void hash_object_file(const struct git_hash_algo *algo, const void *buf,\n+\t\t unsigned long len, enum object_type type,\n+\t\t struct object_id *oid);\n+\n /* Helper to check and \"touch\" a file */\n int check_and_freshen_file(const char *fn, int freshen);\n \ndiff --git a/object-name.c b/object-name.c\nindex 2c751a5352a..9288b2dd245 100644\n--- a/object-name.c\n+++ b/object-name.c\n@@ -19,7 +19,7 @@\n #include \"oidtree.h\"\n #include \"packfile.h\"\n #include \"pretty.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"read-cache-ll.h\"\n #include \"repo-settings.h\"\n #include \"repository.h\"\ndiff --git a/object-store.h b/object-store.h\nindex aa8fc63043e..9dc39a7c91e 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -82,16 +82,6 @@ struct object_directory *set_temporary_primary_odb(const char *dir, int will_des\n */\n void restore_primary_odb(struct object_directory *restore_odb, const char *old_path);\n \n-/*\n- * Populate and return the loose object cache array corresponding to the\n- * given object ID.\n- */\n-struct oidtree *odb_loose_cache(struct object_directory *odb,\n-\t\t\t\t const struct object_id *oid);\n-\n-/* Empty the loose object cache for the specified object directory. */\n-void odb_clear_loose_cache(struct object_directory *odb);\n-\n struct packed_git;\n struct multi_pack_index;\n struct cached_object_entry;\n@@ -189,9 +179,6 @@ void raw_object_store_clear(struct raw_object_store *o);\n */\n int odb_mkstemp(struct strbuf *temp_filename, const char *pattern);\n \n-void *map_loose_object(struct repository *r, const struct object_id *oid,\n-\t\t unsigned long *size);\n-\n void *repo_read_object_file(struct repository *r,\n \t\t\t const struct object_id *oid,\n \t\t\t enum object_type *type,\n@@ -200,10 +187,6 @@ void *repo_read_object_file(struct repository *r,\n /* Read and unpack an object file into memory, write memory to an object file */\n int oid_object_info(struct repository *r, const struct object_id *, unsigned long *);\n \n-void hash_object_file(const struct git_hash_algo *algo, const void *buf,\n-\t\t unsigned long len, enum object_type type,\n-\t\t struct object_id *oid);\n-\n /*\n * Add an object file to the in-memory object store, without writing it\n * to disk.\n@@ -340,56 +323,7 @@ static inline void obj_read_unlock(void)\n \tif(obj_read_use_lock)\n \t\tpthread_mutex_unlock(&obj_read_mutex);\n }\n-\n-/*\n- * Iterate over the files in the loose-object parts of the object\n- * directory \"path\", triggering the following callbacks:\n- *\n- * - loose_object is called for each loose object we find.\n- *\n- * - loose_cruft is called for any files that do not appear to be\n- * loose objects. Note that we only look in the loose object\n- * directories \"objects/[0-9a-f]{2}/\", so we will not report\n- * \"objects/foobar\" as cruft.\n- *\n- * - loose_subdir is called for each top-level hashed subdirectory\n- * of the object directory (e.g., \"$OBJDIR/f0\"). It is called\n- * after the objects in the directory are processed.\n- *\n- * Any callback that is NULL will be ignored. Callbacks returning non-zero\n- * will end the iteration.\n- *\n- * In the \"buf\" variant, \"path\" is a strbuf which will also be used as a\n- * scratch buffer, but restored to its original contents before\n- * the function returns.\n- */\n-typedef int each_loose_object_fn(const struct object_id *oid,\n-\t\t\t\t const char *path,\n-\t\t\t\t void *data);\n-typedef int each_loose_cruft_fn(const char *basename,\n-\t\t\t\tconst char *path,\n-\t\t\t\tvoid *data);\n-typedef int each_loose_subdir_fn(unsigned int nr,\n-\t\t\t\t const char *path,\n-\t\t\t\t void *data);\n-int for_each_file_in_obj_subdir(unsigned int subdir_nr,\n-\t\t\t\tstruct strbuf *path,\n-\t\t\t\teach_loose_object_fn obj_cb,\n-\t\t\t\teach_loose_cruft_fn cruft_cb,\n-\t\t\t\teach_loose_subdir_fn subdir_cb,\n-\t\t\t\tvoid *data);\n-int for_each_loose_file_in_objdir(const char *path,\n-\t\t\t\t each_loose_object_fn obj_cb,\n-\t\t\t\t each_loose_cruft_fn cruft_cb,\n-\t\t\t\t each_loose_subdir_fn subdir_cb,\n-\t\t\t\t void *data);\n-int for_each_loose_file_in_objdir_buf(struct strbuf *path,\n-\t\t\t\t each_loose_object_fn obj_cb,\n-\t\t\t\t each_loose_cruft_fn cruft_cb,\n-\t\t\t\t each_loose_subdir_fn subdir_cb,\n-\t\t\t\t void *data);\n-\n-/* Flags for for_each_*_object() below. */\n+/* Flags for for_each_*_object(). */\n enum for_each_object_flags {\n \t/* Iterate only over local objects, not alternates. */\n \tFOR_EACH_OBJECT_LOCAL_ONLY = (1<<0),\n@@ -409,33 +343,6 @@ enum for_each_object_flags {\n \tFOR_EACH_OBJECT_SKIP_ON_DISK_KEPT_PACKS = (1<<4),\n };\n \n-/*\n- * Iterate over all accessible loose objects without respect to\n- * reachability. By default, this includes both local and alternate objects.\n- * The order in which objects are visited is unspecified.\n- *\n- * Any flags specific to packs are ignored.\n- */\n-int for_each_loose_object(each_loose_object_fn, void *,\n-\t\t\t enum for_each_object_flags flags);\n-\n-/*\n- * Iterate over all accessible packed objects without respect to reachability.\n- * By default, this includes both local and alternate packs.\n- *\n- * Note that some objects may appear twice if they are found in multiple packs.\n- * Each pack is visited in an unspecified order. By default, objects within a\n- * pack are visited in pack-idx order (i.e., sorted by oid).\n- */\n-typedef int each_packed_object_fn(const struct object_id *oid,\n-\t\t\t\t struct packed_git *pack,\n-\t\t\t\t uint32_t pos,\n-\t\t\t\t void *data);\n-int for_each_object_in_pack(struct packed_git *p,\n-\t\t\t each_packed_object_fn, void *data,\n-\t\t\t enum for_each_object_flags flags);\n-int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,\n-\t\t\t void *data, enum for_each_object_flags flags);\n \n void *read_object_with_reference(struct repository *r,\n \t\t\t\t const struct object_id *oid,\ndiff --git a/packfile.h b/packfile.h\nindex 05499382397..3a3c77cf05a 100644\n--- a/packfile.h\n+++ b/packfile.h\n@@ -3,6 +3,7 @@\n \n #include \"list.h\"\n #include \"object.h\"\n+#include \"object-store.h\"\n #include \"oidset.h\"\n \n /* in object-store.h */\n@@ -117,6 +118,24 @@ void for_each_file_in_pack_dir(const char *objdir,\n \t\t\t each_file_in_pack_dir_fn fn,\n \t\t\t void *data);\n \n+/*\n+ * Iterate over all accessible packed objects without respect to reachability.\n+ * By default, this includes both local and alternate packs.\n+ *\n+ * Note that some objects may appear twice if they are found in multiple packs.\n+ * Each pack is visited in an unspecified order. By default, objects within a\n+ * pack are visited in pack-idx order (i.e., sorted by oid).\n+ */\n+typedef int each_packed_object_fn(const struct object_id *oid,\n+\t\t\t\t struct packed_git *pack,\n+\t\t\t\t uint32_t pos,\n+\t\t\t\t void *data);\n+int for_each_object_in_pack(struct packed_git *p,\n+\t\t\t each_packed_object_fn, void *data,\n+\t\t\t enum for_each_object_flags flags);\n+int for_each_packed_object(struct repository *repo, each_packed_object_fn cb,\n+\t\t\t void *data, enum for_each_object_flags flags);\n+\n /* A hook to report invalid files in pack directory */\n #define PACKDIR_FILE_PACK 1\n #define PACKDIR_FILE_IDX 2\ndiff --git a/prune-packed.c b/prune-packed.c\nindex c1d95a519d7..92fb4fbb0ed 100644\n--- a/prune-packed.c\n+++ b/prune-packed.c\n@@ -2,7 +2,7 @@\n \n #include \"git-compat-util.h\"\n #include \"gettext.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"packfile.h\"\n #include \"progress.h\"\n #include \"prune-packed.h\"\ndiff --git a/reachable.c b/reachable.c\nindex e5f56f40181..9dc748f0b9a 100644\n--- a/reachable.c\n+++ b/reachable.c\n@@ -14,7 +14,7 @@\n #include \"list-objects.h\"\n #include \"packfile.h\"\n #include \"worktree.h\"\n-#include \"object-store.h\"\n+#include \"object-file.h\"\n #include \"pack-bitmap.h\"\n #include \"pack-mtimes.h\"\n #include \"config.h\"\n\n-- \n2.49.0.967.g6a0df3ecc3.dirty\n\n"},{"id":"516938","messageId":"20250429-pks-object-store-cleanups-v3-5-42902bad09a1@pks.im","threadId":"63335","inReplyTo":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","subject":"[PATCH v3 5/7] object-store: allow fetching objects via `has_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T07:52:19Z","receivedAt":"2025-04-29T07:52:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"We're about to fully remove `repo_has_object_file()` in favor of\n`has_object()`. The latter function does not yet have a way to fetch\nmissing objects via a promisor remote though, which means that it cannot\nfully replace all usecases of `repo_has_object_file()`.\n\nIntroduce a new flag `HAS_OBJECT_FETCH_PROMISOR` that causes the\nfunction to optionally fetch missing objects which are part of a\npromisor pack. This flag will be used in the subsequent commit.\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.c | 9 ++++++---\n object-store.h | 10 +++++++---\n 2 files changed, 13 insertions(+), 6 deletions(-)\n\ndiff --git a/object-store.c b/object-store.c\nindex 0cbad5a19a0..0d873868a6d 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -937,12 +937,15 @@ void *read_object_with_reference(struct repository *r,\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags)\n {\n-\tint quick = !(flags & HAS_OBJECT_RECHECK_PACKED);\n-\tunsigned object_info_flags = OBJECT_INFO_SKIP_FETCH_OBJECT |\n-\t\t(quick ? OBJECT_INFO_QUICK : 0);\n+\tunsigned object_info_flags = 0;\n \n \tif (!startup_info->have_repository)\n \t\treturn 0;\n+\tif (!(flags & HAS_OBJECT_RECHECK_PACKED))\n+\t\tobject_info_flags |= OBJECT_INFO_QUICK;\n+\tif (!(flags & HAS_OBJECT_FETCH_PROMISOR))\n+\t\tobject_info_flags |= OBJECT_INFO_SKIP_FETCH_OBJECT;\n+\n \treturn oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;\n }\n \ndiff --git a/object-store.h b/object-store.h\nindex 9dc39a7c91e..f0e111464c2 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -262,12 +262,16 @@ int oid_object_info_extended(struct repository *r,\n \t\t\t const struct object_id *,\n \t\t\t struct object_info *, unsigned flags);\n \n-/* Retry packed storage after checking packed and loose storage */\n-#define HAS_OBJECT_RECHECK_PACKED 1\n+enum {\n+\t/* Retry packed storage after checking packed and loose storage */\n+\tHAS_OBJECT_RECHECK_PACKED = (1 << 0),\n+\t/* Allow fetching the object in case the repository has a promisor remote. */\n+\tHAS_OBJECT_FETCH_PROMISOR = (1 << 1),\n+};\n \n /*\n * Returns 1 if the object exists. This function will not lazily fetch objects\n- * in a partial clone.\n+ * in a partial clone by default.\n */\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags);\n\n-- \n2.49.0.967.g6a0df3ecc3.dirty\n\n"},{"id":"516939","messageId":"20250429-pks-object-store-cleanups-v3-6-42902bad09a1@pks.im","threadId":"63335","inReplyTo":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","subject":"[PATCH v3 6/7] treewide: convert users of `repo_has_object_file()` to `has_object()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T07:52:20Z","receivedAt":"2025-04-29T07:52:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"As the comment of `repo_has_object_file()` and its `_with_flags()`\nvariant tells us, these functions are considered to be deprecated in\nfavor of `has_object()`. There are a couple of slight benefits in favor\nof the replacement:\n\n - The new function has a short-and-sweet name.\n\n - More explicit defaults: `has_object()` doesn't fetch missing objects\n via promisor remotes, and neither does it reload packfiles if an\n object wasn't found by default. This ensures that it becomes\n immediately obvious when a simple object existence check may result\n in expensive actions.\n\nMost importantly though, it is confusing that we have two sets of\nfunctions that ultimately do the same thing, but with different\ndefaults.\n\nStart sunsetting `repo_has_object_file()` and its `_with_flags()`\nsibling by replacing all callsites with `has_object()`:\n\n - `repo_has_object_file(...)` is equivalent to\n `has_object(..., HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)`.\n\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT)`\n is equivalent to `has_object(..., 0)`.\n\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_SKIP_FETCH_OBJECT)`\n is equivalent to `has_object(..., HAS_OBJECT_RECHECK_PACKED)`.\n\n - `repo_has_object_file_with_flags(..., OBJECT_INFO_QUICK)`\n is equivalent to `has_object(..., HAS_OBJECT_FETCH_PROMISOR)`.\n\nThe replacements should be functionally equivalent.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/cat-file.c | 3 ++-\n builtin/clone.c | 4 +---\n builtin/fetch.c | 15 +++++++--------\n builtin/index-pack.c | 5 ++---\n builtin/receive-pack.c | 4 +++-\n builtin/remote.c | 3 ++-\n builtin/show-ref.c | 3 ++-\n builtin/unpack-objects.c | 3 ++-\n bulk-checkin.c | 3 ++-\n cache-tree.c | 13 +++++++++----\n fetch-pack.c | 7 +++----\n http-push.c | 11 +++++++----\n http-walker.c | 6 ++++--\n list-objects.c | 3 ++-\n notes.c | 3 ++-\n object-store.c | 2 +-\n reflog.c | 3 ++-\n refs.c | 2 +-\n remote.c | 2 +-\n send-pack.c | 5 +----\n shallow.c | 9 ++++++---\n upload-pack.c | 3 +--\n walker.c | 3 ++-\n 23 files changed, 65 insertions(+), 50 deletions(-)\n\ndiff --git a/builtin/cat-file.c b/builtin/cat-file.c\nindex 0e3f10a9467..3914a2a3f61 100644\n--- a/builtin/cat-file.c\n+++ b/builtin/cat-file.c\n@@ -169,7 +169,8 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name,\n \t\tgoto cleanup;\n \n \tcase 'e':\n-\t\tret = !repo_has_object_file(the_repository, &oid);\n+\t\tret = !has_object(the_repository, &oid,\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR);\n \t\tgoto cleanup;\n \n \tcase 'w':\ndiff --git a/builtin/clone.c b/builtin/clone.c\nindex 6b1d11a3ed2..b498b81a043 100644\n--- a/builtin/clone.c\n+++ b/builtin/clone.c\n@@ -504,9 +504,7 @@ static void write_followtags(const struct ref *refs, const char *msg)\n \t\t\tcontinue;\n \t\tif (ends_with(ref->name, \"^{}\"))\n \t\t\tcontinue;\n-\t\tif (!repo_has_object_file_with_flags(the_repository, &ref->old_oid,\n-\t\t\t\t\t\t OBJECT_INFO_QUICK |\n-\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (!has_object(the_repository, &ref->old_oid, 0))\n \t\t\tcontinue;\n \t\trefs_update_ref(get_main_ref_store(the_repository), msg,\n \t\t\t\tref->name, &ref->old_oid, NULL, 0,\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex 95589b49948..aadcf49a5b4 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -337,7 +337,6 @@ static void find_non_local_tags(const struct ref *refs,\n \tstruct string_list_item *remote_ref_item;\n \tconst struct ref *ref;\n \tstruct refname_hash_entry *item = NULL;\n-\tconst int quick_flags = OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT;\n \n \trefname_hash_init(&existing_refs);\n \trefname_hash_init(&remote_refs);\n@@ -367,9 +366,9 @@ static void find_non_local_tags(const struct ref *refs,\n \t\t */\n \t\tif (ends_with(ref->name, \"^{}\")) {\n \t\t\tif (item &&\n-\t\t\t !repo_has_object_file_with_flags(the_repository, &ref->old_oid, quick_flags) &&\n+\t\t\t !has_object(the_repository, &ref->old_oid, 0) &&\n \t\t\t !oidset_contains(&fetch_oids, &ref->old_oid) &&\n-\t\t\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t\t\t !has_object(the_repository, &item->oid, 0) &&\n \t\t\t !oidset_contains(&fetch_oids, &item->oid))\n \t\t\t\tclear_item(item);\n \t\t\titem = NULL;\n@@ -383,7 +382,7 @@ static void find_non_local_tags(const struct ref *refs,\n \t\t * fetch.\n \t\t */\n \t\tif (item &&\n-\t\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t\t !has_object(the_repository, &item->oid, 0) &&\n \t\t !oidset_contains(&fetch_oids, &item->oid))\n \t\t\tclear_item(item);\n \n@@ -404,7 +403,7 @@ static void find_non_local_tags(const struct ref *refs,\n \t * checked to see if it needs fetching.\n \t */\n \tif (item &&\n-\t !repo_has_object_file_with_flags(the_repository, &item->oid, quick_flags) &&\n+\t !has_object(the_repository, &item->oid, 0) &&\n \t !oidset_contains(&fetch_oids, &item->oid))\n \t\tclear_item(item);\n \n@@ -911,7 +910,8 @@ static int update_local_ref(struct ref *ref,\n \tstruct commit *current = NULL, *updated;\n \tint fast_forward = 0;\n \n-\tif (!repo_has_object_file(the_repository, &ref->new_oid))\n+\tif (!has_object(the_repository, &ref->new_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\tdie(_(\"object %s not found\"), oid_to_hex(&ref->new_oid));\n \n \tif (oideq(&ref->old_oid, &ref->new_oid)) {\n@@ -1330,8 +1330,7 @@ static int check_exist_and_connected(struct ref *ref_map)\n \t * we need all direct targets to exist.\n \t */\n \tfor (r = rm; r; r = r->next) {\n-\t\tif (!repo_has_object_file_with_flags(the_repository, &r->old_oid,\n-\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (!has_object(the_repository, &r->old_oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\treturn -1;\n \t}\n \ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex f49431d626b..147e9b8b479 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -892,9 +892,8 @@ static void sha1_object(const void *data, struct object_entry *obj_entry,\n \n \tif (startup_info->have_repository) {\n \t\tread_lock();\n-\t\tcollision_test_needed =\n-\t\t\trepo_has_object_file_with_flags(the_repository, oid,\n-\t\t\t\t\t\t\tOBJECT_INFO_QUICK);\n+\t\tcollision_test_needed = has_object(the_repository, oid,\n+\t\t\t\t\t\t HAS_OBJECT_FETCH_PROMISOR);\n \t\tread_unlock();\n \t}\n \ndiff --git a/builtin/receive-pack.c b/builtin/receive-pack.c\nindex be314879e82..c92e57ba188 100644\n--- a/builtin/receive-pack.c\n+++ b/builtin/receive-pack.c\n@@ -1506,7 +1506,9 @@ static const char *update(struct command *cmd, struct shallow_info *si)\n \t\t}\n \t}\n \n-\tif (!is_null_oid(new_oid) && !repo_has_object_file(the_repository, new_oid)) {\n+\tif (!is_null_oid(new_oid) &&\n+\t !has_object(the_repository, new_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\terror(\"unpack should have generated %s, \"\n \t\t \"but I can't find it!\", oid_to_hex(new_oid));\n \t\tret = \"bad pack\";\ndiff --git a/builtin/remote.c b/builtin/remote.c\nindex b4baa34e665..0d6755bcb71 100644\n--- a/builtin/remote.c\n+++ b/builtin/remote.c\n@@ -454,7 +454,8 @@ static int get_push_ref_states(const struct ref *remote_refs,\n \t\t\tinfo->status = PUSH_STATUS_UPTODATE;\n \t\telse if (is_null_oid(&ref->old_oid))\n \t\t\tinfo->status = PUSH_STATUS_CREATE;\n-\t\telse if (repo_has_object_file(the_repository, &ref->old_oid) &&\n+\t\telse if (has_object(the_repository, &ref->old_oid,\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) &&\n \t\t\t ref_newer(&ref->new_oid, &ref->old_oid))\n \t\t\tinfo->status = PUSH_STATUS_FASTFORWARD;\n \t\telse\ndiff --git a/builtin/show-ref.c b/builtin/show-ref.c\nindex f81209f23c3..623a52a45f8 100644\n--- a/builtin/show-ref.c\n+++ b/builtin/show-ref.c\n@@ -35,7 +35,8 @@ static void show_one(const struct show_one_options *opts,\n \tconst char *hex;\n \tstruct object_id peeled;\n \n-\tif (!repo_has_object_file(the_repository, oid))\n+\tif (!has_object(the_repository, oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\tdie(\"git show-ref: bad ref %s (%s)\", refname,\n \t\t oid_to_hex(oid));\n \ndiff --git a/builtin/unpack-objects.c b/builtin/unpack-objects.c\nindex 661be789f13..e905d5f4e19 100644\n--- a/builtin/unpack-objects.c\n+++ b/builtin/unpack-objects.c\n@@ -449,7 +449,8 @@ static void unpack_delta_entry(enum object_type type, unsigned long delta_size,\n \t\tdelta_data = get_data(delta_size);\n \t\tif (!delta_data)\n \t\t\treturn;\n-\t\tif (repo_has_object_file(the_repository, &base_oid))\n+\t\tif (has_object(the_repository, &base_oid,\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\t; /* Ok we have this one */\n \t\telse if (resolve_against_held(nr, &base_oid,\n \t\t\t\t\t delta_data, delta_size))\ndiff --git a/bulk-checkin.c b/bulk-checkin.c\nindex c31c31b18d8..678e2ecc2c2 100644\n--- a/bulk-checkin.c\n+++ b/bulk-checkin.c\n@@ -130,7 +130,8 @@ static void flush_batch_fsync(void)\n static int already_written(struct bulk_checkin_packfile *state, struct object_id *oid)\n {\n \t/* The object may already exist in the repository */\n-\tif (repo_has_object_file(the_repository, oid))\n+\tif (has_object(the_repository, oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn 1;\n \n \t/* Might want to keep the list sorted */\ndiff --git a/cache-tree.c b/cache-tree.c\nindex c0e1e9ee1d4..fa3858e2829 100644\n--- a/cache-tree.c\n+++ b/cache-tree.c\n@@ -238,7 +238,9 @@ int cache_tree_fully_valid(struct cache_tree *it)\n \tint i;\n \tif (!it)\n \t\treturn 0;\n-\tif (it->entry_count < 0 || !repo_has_object_file(the_repository, &it->oid))\n+\tif (it->entry_count < 0 ||\n+\t has_object(the_repository, &it->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn 0;\n \tfor (i = 0; i < it->subtree_nr; i++) {\n \t\tif (!cache_tree_fully_valid(it->down[i]->cache_tree))\n@@ -289,7 +291,9 @@ static int update_one(struct cache_tree *it,\n \t\t}\n \t}\n \n-\tif (0 <= it->entry_count && repo_has_object_file(the_repository, &it->oid))\n+\tif (0 <= it->entry_count &&\n+\t has_object(the_repository, &it->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn it->entry_count;\n \n \t/*\n@@ -395,7 +399,8 @@ static int update_one(struct cache_tree *it,\n \t\tce_missing_ok = mode == S_IFGITLINK || missing_ok ||\n \t\t\t!must_check_existence(ce);\n \t\tif (is_null_oid(oid) ||\n-\t\t (!ce_missing_ok && !repo_has_object_file(the_repository, oid))) {\n+\t\t (!ce_missing_ok && !has_object(the_repository, oid,\n+\t\t\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))) {\n \t\t\tstrbuf_release(&buffer);\n \t\t\tif (expected_missing)\n \t\t\t\treturn -1;\n@@ -443,7 +448,7 @@ static int update_one(struct cache_tree *it,\n \t\tstruct object_id oid;\n \t\thash_object_file(the_hash_algo, buffer.buf, buffer.len,\n \t\t\t\t OBJ_TREE, &oid);\n-\t\tif (repo_has_object_file_with_flags(the_repository, &oid, OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\tif (has_object(the_repository, &oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\toidcpy(&it->oid, &oid);\n \t\telse\n \t\t\tto_invalidate = 1;\ndiff --git a/fetch-pack.c b/fetch-pack.c\nindex 210dc30d50f..fa4231fee74 100644\n--- a/fetch-pack.c\n+++ b/fetch-pack.c\n@@ -769,9 +769,7 @@ static void mark_complete_and_common_ref(struct fetch_negotiator *negotiator,\n \t\tif (!commit) {\n \t\t\tstruct object *o;\n \n-\t\t\tif (!repo_has_object_file_with_flags(the_repository, &ref->old_oid,\n-\t\t\t\t\t\t\t OBJECT_INFO_QUICK |\n-\t\t\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\t\tif (!has_object(the_repository, &ref->old_oid, 0))\n \t\t\t\tcontinue;\n \t\t\to = parse_object(the_repository, &ref->old_oid);\n \t\t\tif (!o || o->type != OBJ_COMMIT)\n@@ -1985,7 +1983,8 @@ static void update_shallow(struct fetch_pack_args *args,\n \t\tstruct oid_array extra = OID_ARRAY_INIT;\n \t\tstruct object_id *oid = si->shallow->oid;\n \t\tfor (i = 0; i < si->shallow->nr; i++)\n-\t\t\tif (repo_has_object_file(the_repository, &oid[i]))\n+\t\t\tif (has_object(the_repository, &oid[i],\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\t\toid_array_append(&extra, &oid[i]);\n \t\tif (extra.nr) {\n \t\t\tsetup_alternate_shallow(&shallow_lock,\ndiff --git a/http-push.c b/http-push.c\nindex 32e37565f4e..f9e67cabd4b 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -1446,7 +1446,9 @@ static void one_remote_ref(const char *refname)\n \t * Fetch a copy of the object if it doesn't exist locally - it\n \t * may be required for updating server info later.\n \t */\n-\tif (repo->can_update_info_refs && !repo_has_object_file(the_repository, &ref->old_oid)) {\n+\tif (repo->can_update_info_refs &&\n+\t !has_object(the_repository, &ref->old_oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\tobj = lookup_unknown_object(the_repository, &ref->old_oid);\n \t\tfprintf(stderr,\t\" fetch %s for %s\\n\",\n \t\t\toid_to_hex(&ref->old_oid), refname);\n@@ -1651,14 +1653,14 @@ static int delete_remote_branch(const char *pattern, int force)\n \t\t\treturn error(\"Remote HEAD symrefs too deep\");\n \t\tif (is_null_oid(&head_oid))\n \t\t\treturn error(\"Unable to resolve remote HEAD\");\n-\t\tif (!repo_has_object_file(the_repository, &head_oid))\n+\t\tif (!has_object(the_repository, &head_oid, HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn error(\"Remote HEAD resolves to object %s\\nwhich does not exist locally, perhaps you need to fetch?\", oid_to_hex(&head_oid));\n \n \t\t/* Remote branch must resolve to a known object */\n \t\tif (is_null_oid(&remote_ref->old_oid))\n \t\t\treturn error(\"Unable to resolve remote branch %s\",\n \t\t\t\t remote_ref->name);\n-\t\tif (!repo_has_object_file(the_repository, &remote_ref->old_oid))\n+\t\tif (!has_object(the_repository, &remote_ref->old_oid, HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn error(\"Remote branch %s resolves to object %s\\nwhich does not exist locally, perhaps you need to fetch?\", remote_ref->name, oid_to_hex(&remote_ref->old_oid));\n \n \t\t/* Remote branch must be an ancestor of remote HEAD */\n@@ -1879,7 +1881,8 @@ int cmd_main(int argc, const char **argv)\n \t\tif (!force_all &&\n \t\t !is_null_oid(&ref->old_oid) &&\n \t\t !ref->force) {\n-\t\t\tif (!repo_has_object_file(the_repository, &ref->old_oid) ||\n+\t\t\tif (!has_object(the_repository, &ref->old_oid,\n+\t\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) ||\n \t\t\t !ref_newer(&ref->peer_ref->new_oid,\n \t\t\t\t &ref->old_oid)) {\n \t\t\t\t/*\ndiff --git a/http-walker.c b/http-walker.c\nindex 95458e2f638..463f7b119ad 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -138,7 +138,8 @@ static int fill_active_slot(void *data UNUSED)\n \tlist_for_each_safe(pos, tmp, head) {\n \t\tobj_req = list_entry(pos, struct object_request, node);\n \t\tif (obj_req->state == WAITING) {\n-\t\t\tif (repo_has_object_file(the_repository, &obj_req->oid))\n+\t\t\tif (has_object(the_repository, &obj_req->oid,\n+\t\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\t\tobj_req->state = COMPLETE;\n \t\t\telse {\n \t\t\t\tstart_object_request(obj_req);\n@@ -496,7 +497,8 @@ static int fetch_object(struct walker *walker, const struct object_id *oid)\n \tif (!obj_req)\n \t\treturn error(\"Couldn't find request for %s in the queue\", hex);\n \n-\tif (repo_has_object_file(the_repository, &obj_req->oid)) {\n+\tif (has_object(the_repository, &obj_req->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\tif (obj_req->req)\n \t\t\tabort_http_object_request(&obj_req->req);\n \t\tabort_object_request(obj_req);\ndiff --git a/list-objects.c b/list-objects.c\nindex 1e5512e1318..597114281f6 100644\n--- a/list-objects.c\n+++ b/list-objects.c\n@@ -74,7 +74,8 @@ static void process_blob(struct traversal_context *ctx,\n \t * of missing objects.\n \t */\n \tif (ctx->revs->exclude_promisor_objects &&\n-\t !repo_has_object_file(the_repository, &obj->oid) &&\n+\t !has_object(the_repository, &obj->oid,\n+\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) &&\n \t is_promisor_object(ctx->revs->repo, &obj->oid))\n \t\treturn;\n \ndiff --git a/notes.c b/notes.c\nindex d9645c4b5dc..0a128f1de98 100644\n--- a/notes.c\n+++ b/notes.c\n@@ -794,7 +794,8 @@ static int prune_notes_helper(const struct object_id *object_oid,\n \tstruct note_delete_list **l = (struct note_delete_list **) cb_data;\n \tstruct note_delete_list *n;\n \n-\tif (repo_has_object_file(the_repository, object_oid))\n+\tif (has_object(the_repository, object_oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\treturn 0; /* nothing to do for this note */\n \n \t/* failed to find object => prune this note */\ndiff --git a/object-store.c b/object-store.c\nindex 0d873868a6d..2db34804e8f 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -847,7 +847,7 @@ int pretend_object_file(struct repository *repo,\n \tchar *co_buf;\n \n \thash_object_file(repo->hash_algo, buf, len, type, oid);\n-\tif (repo_has_object_file_with_flags(repo, oid, OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT) ||\n+\tif (has_object(repo, oid, 0) ||\n \t find_cached_object(repo->objects, oid))\n \t\treturn 0;\n \ndiff --git a/reflog.c b/reflog.c\nindex c6c534dfad4..2db1cc81d01 100644\n--- a/reflog.c\n+++ b/reflog.c\n@@ -152,7 +152,8 @@ static int tree_is_complete(const struct object_id *oid)\n \tinit_tree_desc(&desc, &tree->object.oid, tree->buffer, tree->size);\n \tcomplete = 1;\n \twhile (tree_entry(&desc, &entry)) {\n-\t\tif (!repo_has_object_file(the_repository, &entry.oid) ||\n+\t\tif (!has_object(the_repository, &entry.oid,\n+\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR) ||\n \t\t (S_ISDIR(entry.mode) && !tree_is_complete(&entry.oid))) {\n \t\t\ttree->object.flags |= INCOMPLETE;\n \t\t\tcomplete = 0;\ndiff --git a/refs.c b/refs.c\nindex 6559db37890..dce5c49ca2b 100644\n--- a/refs.c\n+++ b/refs.c\n@@ -376,7 +376,7 @@ int ref_resolves_to_object(const char *refname,\n {\n \tif (flags & REF_ISBROKEN)\n \t\treturn 0;\n-\tif (!repo_has_object_file(repo, oid)) {\n+\tif (!has_object(repo, oid, HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\terror(_(\"%s does not point to a valid object!\"), refname);\n \t\treturn 0;\n \t}\ndiff --git a/remote.c b/remote.c\nindex 9fa3614e7a3..4099183cacd 100644\n--- a/remote.c\n+++ b/remote.c\n@@ -1702,7 +1702,7 @@ void set_ref_status_for_push(struct ref *remote_refs, int send_mirror,\n \t\tif (!reject_reason && !ref->deletion && !is_null_oid(&ref->old_oid)) {\n \t\t\tif (starts_with(ref->name, \"refs/tags/\"))\n \t\t\t\treject_reason = REF_STATUS_REJECT_ALREADY_EXISTS;\n-\t\t\telse if (!repo_has_object_file_with_flags(the_repository, &ref->old_oid, OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\t\t\telse if (!has_object(the_repository, &ref->old_oid, HAS_OBJECT_RECHECK_PACKED))\n \t\t\t\treject_reason = REF_STATUS_REJECT_FETCH_FIRST;\n \t\t\telse if (!lookup_commit_reference_gently(the_repository, &ref->old_oid, 1) ||\n \t\t\t\t !lookup_commit_reference_gently(the_repository, &ref->new_oid, 1))\ndiff --git a/send-pack.c b/send-pack.c\nindex 5005689cb55..86592ce526d 100644\n--- a/send-pack.c\n+++ b/send-pack.c\n@@ -45,10 +45,7 @@ int option_parse_push_signed(const struct option *opt,\n static void feed_object(struct repository *r,\n \t\t\tconst struct object_id *oid, FILE *fh, int negative)\n {\n-\tif (negative &&\n-\t !repo_has_object_file_with_flags(r, oid,\n-\t\t\t\t\t OBJECT_INFO_SKIP_FETCH_OBJECT |\n-\t\t\t\t\t OBJECT_INFO_QUICK))\n+\tif (negative && !has_object(r, oid, 0))\n \t\treturn;\n \n \tif (negative)\ndiff --git a/shallow.c b/shallow.c\nindex 2f82ebd6e3f..faeeeb45f98 100644\n--- a/shallow.c\n+++ b/shallow.c\n@@ -310,7 +310,8 @@ static int write_one_shallow(const struct commit_graft *graft, void *cb_data)\n \tif (graft->nr_parent != -1)\n \t\treturn 0;\n \tif (data->flags & QUICK) {\n-\t\tif (!repo_has_object_file(the_repository, &graft->oid))\n+\t\tif (!has_object(the_repository, &graft->oid,\n+\t\t\t\tHAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\treturn 0;\n \t} else if (data->flags & SEEN_ONLY) {\n \t\tstruct commit *c = lookup_commit(the_repository, &graft->oid);\n@@ -476,7 +477,8 @@ void prepare_shallow_info(struct shallow_info *info, struct oid_array *sa)\n \tALLOC_ARRAY(info->ours, sa->nr);\n \tALLOC_ARRAY(info->theirs, sa->nr);\n \tfor (size_t i = 0; i < sa->nr; i++) {\n-\t\tif (repo_has_object_file(the_repository, sa->oid + i)) {\n+\t\tif (has_object(the_repository, sa->oid + i,\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\t\tstruct commit_graft *graft;\n \t\t\tgraft = lookup_commit_graft(the_repository,\n \t\t\t\t\t\t &sa->oid[i]);\n@@ -513,7 +515,8 @@ void remove_nonexistent_theirs_shallow(struct shallow_info *info)\n \tfor (i = dst = 0; i < info->nr_theirs; i++) {\n \t\tif (i != dst)\n \t\t\tinfo->theirs[dst] = info->theirs[i];\n-\t\tif (repo_has_object_file(the_repository, oid + info->theirs[i]))\n+\t\tif (has_object(the_repository, oid + info->theirs[i],\n+\t\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR))\n \t\t\tdst++;\n \t}\n \tinfo->nr_theirs = dst;\ndiff --git a/upload-pack.c b/upload-pack.c\nindex 30e4630f3a1..956da5b061a 100644\n--- a/upload-pack.c\n+++ b/upload-pack.c\n@@ -509,8 +509,7 @@ static int got_oid(struct upload_pack_data *data,\n {\n \tif (get_oid_hex(hex, oid))\n \t\tdie(\"git upload-pack: expected SHA1 object, got '%s'\", hex);\n-\tif (!repo_has_object_file_with_flags(the_repository, oid,\n-\t\t\t\t\t OBJECT_INFO_QUICK | OBJECT_INFO_SKIP_FETCH_OBJECT))\n+\tif (!has_object(the_repository, oid, 0))\n \t\treturn -1;\n \treturn do_got_oid(data, oid);\n }\ndiff --git a/walker.c b/walker.c\nindex 4fedc19f346..b470d43e54d 100644\n--- a/walker.c\n+++ b/walker.c\n@@ -150,7 +150,8 @@ static int process(struct walker *walker, struct object *obj)\n \t\treturn 0;\n \tobj->flags |= SEEN;\n \n-\tif (repo_has_object_file(the_repository, &obj->oid)) {\n+\tif (has_object(the_repository, &obj->oid,\n+\t\t HAS_OBJECT_RECHECK_PACKED | HAS_OBJECT_FETCH_PROMISOR)) {\n \t\t/* We already have it, so we should scan it now. */\n \t\tobj->flags |= TO_SCAN;\n \t}\n\n-- \n2.49.0.967.g6a0df3ecc3.dirty\n\n"},{"id":"516940","messageId":"20250429-pks-object-store-cleanups-v3-7-42902bad09a1@pks.im","threadId":"63335","inReplyTo":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","subject":"[PATCH v3 7/7] object-store: drop `repo_has_object_file()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-04-29T07:52:21Z","receivedAt":"2025-04-29T07:52:30Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In the preceding commits we have converted all users of\n`repo_has_object_file()` and its `_with_flags()` variant to instead use\n`has_object()`. Drop these functions.\n\nSigned-off-by: Patrick Steinhardt \n---\n object-store.c | 14 --------------\n object-store.h | 17 -----------------\n 2 files changed, 31 deletions(-)\n\ndiff --git a/object-store.c b/object-store.c\nindex 2db34804e8f..2f51d0e3b03 100644\n--- a/object-store.c\n+++ b/object-store.c\n@@ -949,20 +949,6 @@ int has_object(struct repository *r, const struct object_id *oid,\n \treturn oid_object_info_extended(r, oid, NULL, object_info_flags) >= 0;\n }\n \n-int repo_has_object_file_with_flags(struct repository *r,\n-\t\t\t\t const struct object_id *oid, int flags)\n-{\n-\tif (!startup_info->have_repository)\n-\t\treturn 0;\n-\treturn oid_object_info_extended(r, oid, NULL, flags) >= 0;\n-}\n-\n-int repo_has_object_file(struct repository *r,\n-\t\t\t const struct object_id *oid)\n-{\n-\treturn repo_has_object_file_with_flags(r, oid, 0);\n-}\n-\n void assert_oid_type(const struct object_id *oid, enum object_type expect)\n {\n \tenum object_type type = oid_object_info(the_repository, oid, NULL);\ndiff --git a/object-store.h b/object-store.h\nindex f0e111464c2..c2fe5a19605 100644\n--- a/object-store.h\n+++ b/object-store.h\n@@ -276,23 +276,6 @@ enum {\n int has_object(struct repository *r, const struct object_id *oid,\n \t unsigned flags);\n \n-/*\n- * These macros and functions are deprecated. If checking existence for an\n- * object that is likely to be missing and/or whose absence is relatively\n- * inconsequential (or is consequential but the caller is prepared to handle\n- * it), use has_object(), which has better defaults (no lazy fetch in a partial\n- * clone and no rechecking of packed storage). In the unlikely event that a\n- * caller needs to assert existence of an object that it fully expects to\n- * exist, and wants to trigger a lazy fetch in a partial clone, use\n- * oid_object_info_extended() with a NULL struct object_info.\n- *\n- * These functions can be removed once all callers have migrated to\n- * has_object() and/or oid_object_info_extended().\n- */\n-int repo_has_object_file(struct repository *r, const struct object_id *oid);\n-int repo_has_object_file_with_flags(struct repository *r,\n-\t\t\t\t const struct object_id *oid, int flags);\n-\n void assert_oid_type(const struct object_id *oid, enum object_type expect);\n \n /*\n\n-- \n2.49.0.967.g6a0df3ecc3.dirty\n\n"},{"id":"516958","messageId":"xmqq5xin9ehg.fsf@gitster.g","threadId":"63335","inReplyTo":"aBBukgbWZRRLgnLl@pks.im","subject":"Re: [PATCH v2 12/13] bulk-checkin: don't fetch promised objects on write","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-29T15:25:31Z","receivedAt":"2025-04-29T15:25:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Yeah, to be honest I wasn't totally sure whether to include these steps\n> myself as I anticipated that they will lead to discussions that derail\n> my original goal, which is to clean up the interfaces in the object\n> subsystem. I decided to go with these where I thought that my train of\n> thought is reasonable, but given your comments I'll probably just drop\n> those patches.\n>\n> We can still adapt these callsites in the future as needed.\n\nIt is probably why among our past rewrites and refactors, successful\nones started with a rewrite faithful to the original, treating\nsemantic improvements as a set of separate topics on top. I think\nsome semantic-improvement steps like the http walker one make sense,\nbut even for them, others may give us some reasons why it is not a\ngood idea to cause me change my mind. On the other hand, steps I\nthought whose semantic changes were undesirable may turn out to be\nbenign (they unquestionably will improve the performance of the call\npath; it was quesitonable to me if they still gave us correct\nresults). So let's give people some time to evaluate. I'd consider\nthese \"improved behaviour\" changes as \"to be discarded by default,\nunless there are strong supporting arguments why the specific\nsemantic changes are good\" material.\n\nThanks.\n\n"},{"id":"516986","messageId":"xmqq8qni68af.fsf@gitster.g","threadId":"63335","inReplyTo":"20250429-pks-object-store-cleanups-v3-0-42902bad09a1@pks.im","subject":"Re: [PATCH v3 0/7] object-store: a handful of cleanups","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-04-29T20:07:36Z","receivedAt":"2025-04-29T20:07:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Changes in v3:\n> - Move around `hash_object_file()`, which I missed in previous\n> iterations.\n> - Don't try to fix up callsites where we end up fetching promised\n> objects. This patch series now only does a trivial 1:1 conversion.\n> - Clarify why we're sunsetting `repo_has_object_file()`.\n> - Link to v2: https://lore.kernel.org/r/20250425-pks-object-store-cleanups-v2-0-63f1695b7700@pks.im\n\nThanks for a quick reroll. I think these are all good.\n\n"}]}