{"thread":{"id":"63871","subject":"[PATCH 0/8] midx: stop deduplicating info redundant with their sources","startedAt":"2025-07-29T14:12:48Z","lastAt":"2025-09-02T06:36:21Z","messageCount":56,"participants":["Patrick Steinhardt","Junio C Hamano","Toon Claes","Karthik Nayak","Taylor Blau","Derrick Stolee"],"isPatch":true,"patchVersion":1,"patchTotal":8},"messages":[{"id":"522953","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":null,"subject":"[PATCH 0/8] midx: stop deduplicating info redundant with their sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:39Z","receivedAt":"2025-07-29T14:12:48Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series is a follow up to the recent patch series that started\nto move the MIDX into its object source. It refactors the MIDX subsystem\nso that we always read and write them via an ODB source. This ultimately\nallows us to store a pointer to the owning ODB source in the MIDX and\nthus dedup some redundant information.\n\nThis series is built on top of e813a0200a7 (The fifteenth batch,\n2025-07-28) with ps/object-store-midx at ec865d94d46 (midx: remove\nnow-unused linked list of multi-pack indices, 2025-07-15) merged into\nit.\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (8):\n odb: store locality in object database sources\n odb: allow `odb_find_source()` to fail\n odb: return newly created in-memory sources\n midx: drop redundant `struct repository` parameter\n midx: load multi-pack indices via their source\n midx: write multi-pack indices via their source\n midx: stop duplicating info redundant with its owning source\n midx: compute paths via their source\n\n builtin/commit-graph.c | 4 ++\n builtin/multi-pack-index.c | 31 +++++++---\n builtin/pack-objects.c | 2 +-\n builtin/repack.c | 7 ++-\n midx-write.c | 116 +++++++++++++++++--------------------\n midx.c | 135 +++++++++++++++++++++-----------------------\n midx.h | 42 ++++++--------\n odb.c | 40 +++++++------\n odb.h | 16 ++++--\n pack-bitmap.c | 15 +++--\n pack-revindex.c | 14 ++---\n packfile.c | 13 ++---\n repository.c | 1 +\n t/helper/test-read-midx.c | 31 +++++-----\n t/t5319-multi-pack-index.sh | 8 +--\n 15 files changed, 243 insertions(+), 232 deletions(-)\n\n\n---\nbase-commit: ee2fd3d4b224c8145eeff04101ea6075560973e1\nchange-id: 20250729-b4-pks-midx-deduplicate-source-info-fcf3c08bfd53\n\n"},{"id":"522954","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-1-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH 1/8] odb: store locality in object database sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:40Z","receivedAt":"2025-07-29T14:12:50Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Object database sources are classified either as:\n\n - Local, which means that the source is the repository's primary\n source. This is typically \".git/objects\".\n\n - Non-local, which is everything else. Most importantly this includes\n alternates and quarantine directories.\n\nThis locality is often computed ad-hoc by checking whether a given\nobject source is the first one. This works, but it is quite roundabout.\n\nRefactor the code so that we store locality when creating the sources in\nthe first place. This makes it both more accessible and robust.\n\nSigned-off-by: Patrick Steinhardt \n---\n midx.c | 5 +++--\n midx.h | 2 +-\n odb.c | 1 +\n odb.h | 8 ++++++++\n packfile.c | 9 ++++-----\n repository.c | 1 +\n 6 files changed, 18 insertions(+), 8 deletions(-)\n\ndiff --git a/midx.c b/midx.c\nindex 7d407682e6..b9ca0915a6 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -723,7 +723,7 @@ int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id)\n \treturn 0;\n }\n \n-int prepare_multi_pack_index_one(struct odb_source *source, int local)\n+int prepare_multi_pack_index_one(struct odb_source *source)\n {\n \tstruct repository *r = source->odb->repo;\n \n@@ -734,7 +734,8 @@ int prepare_multi_pack_index_one(struct odb_source *source, int local)\n \tif (source->midx)\n \t\treturn 1;\n \n-\tsource->midx = load_multi_pack_index(r, source->path, local);\n+\tsource->midx = load_multi_pack_index(r, source->path,\n+\t\t\t\t\t source->local);\n \n \treturn !!source->midx;\n }\ndiff --git a/midx.h b/midx.h\nindex 076382de8a..28c426a823 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -122,7 +122,7 @@ int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pa\n int midx_contains_pack(struct multi_pack_index *m,\n \t\t const char *idx_or_pack_name);\n int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);\n-int prepare_multi_pack_index_one(struct odb_source *source, int local);\n+int prepare_multi_pack_index_one(struct odb_source *source);\n \n /*\n * Variant of write_midx_file which writes a MIDX containing only the packs\ndiff --git a/odb.c b/odb.c\nindex 1f48a0448e..1761a50840 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -176,6 +176,7 @@ static int link_alt_odb_entry(struct object_database *odb,\n \n \tCALLOC_ARRAY(alternate, 1);\n \talternate->odb = odb;\n+\talternate->local = false;\n \t/* pathbuf.buf is already in r->objects->source_by_path */\n \talternate->path = strbuf_detach(&pathbuf, NULL);\n \ndiff --git a/odb.h b/odb.h\nindex 09177bf430..d9f4dcf79f 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -63,6 +63,14 @@ struct odb_source {\n \t */\n \tstruct multi_pack_index *midx;\n \n+\t/*\n+\t * Figure out whether this is the local alternate of the owning\n+\t * repository, which would typically be its \".git/objects\" directory.\n+\t * This local object directory is usually where objects would be\n+\t * written to.\n+\t */\n+\tbool local;\n+\n \t/*\n \t * This is a temporary object store created by the tmp_objdir\n \t * facility. Disable ref updates since the objects in the store\ndiff --git a/packfile.c b/packfile.c\nindex 5d73932f50..a38544b87b 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -935,14 +935,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,\n \t\treport_garbage(PACKDIR_FILE_GARBAGE, full_name);\n }\n \n-static void prepare_packed_git_one(struct odb_source *source, int local)\n+static void prepare_packed_git_one(struct odb_source *source)\n {\n \tstruct string_list garbage = STRING_LIST_INIT_DUP;\n \tstruct prepare_pack_data data = {\n \t\t.m = source->midx,\n \t\t.r = source->odb->repo,\n \t\t.garbage = &garbage,\n-\t\t.local = local,\n+\t\t.local = source->local,\n \t};\n \n \tfor_each_file_in_pack_dir(source->path, prepare_pack, &data);\n@@ -1037,9 +1037,8 @@ static void prepare_packed_git(struct repository *r)\n \n \todb_prepare_alternates(r->objects);\n \tfor (source = r->objects->sources; source; source = source->next) {\n-\t\tint local = (source == r->objects->sources);\n-\t\tprepare_multi_pack_index_one(source, local);\n-\t\tprepare_packed_git_one(source, local);\n+\t\tprepare_multi_pack_index_one(source);\n+\t\tprepare_packed_git_one(source);\n \t}\n \trearrange_packed_git(r);\n \ndiff --git a/repository.c b/repository.c\nindex ecd691181f..97f0578381 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -168,6 +168,7 @@ void repo_set_gitdir(struct repository *repo,\n \tif (!repo->objects->sources) {\n \t\tCALLOC_ARRAY(repo->objects->sources, 1);\n \t\trepo->objects->sources->odb = repo->objects;\n+\t\trepo->objects->sources->local = true;\n \t\trepo->objects->sources_tail = &repo->objects->sources->next;\n \t}\n \texpand_base_dir(&repo->objects->sources->path, o->object_dir,\n\n-- \n2.50.1.619.g074bbf1d35.dirty\n\n"},{"id":"522955","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-2-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH 2/8] odb: allow `odb_find_source()` to fail","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:41Z","receivedAt":"2025-07-29T14:12:52Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When trying to locate a source for an unknown object directory we will\ndie right away. In subsequent patches we will add new callsites though\nthat want to handle this situation gracefully instead.\n\nRefactor the function to return a `NULL` pointer if the source could not\nbe found and adapt the callsites to die instead.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/commit-graph.c | 4 ++++\n midx-write.c | 2 ++\n odb.c | 2 --\n odb.h | 4 ++--\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\nindex 25018a0b9d..dc2c1a5432 100644\n--- a/builtin/commit-graph.c\n+++ b/builtin/commit-graph.c\n@@ -102,6 +102,8 @@ static int graph_verify(int argc, const char **argv, const char *prefix,\n \t\tflags |= COMMIT_GRAPH_WRITE_PROGRESS;\n \n \tsource = odb_find_source(the_repository->objects, opts.obj_dir);\n+\tif (!source)\n+\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n \tgraph_name = get_commit_graph_filename(source);\n \tchain_name = get_commit_graph_chain_filename(source);\n \tif (open_commit_graph(graph_name, &fd, &st))\n@@ -290,6 +292,8 @@ static int graph_write(int argc, const char **argv, const char *prefix,\n \t\tflags |= COMMIT_GRAPH_WRITE_BLOOM_FILTERS;\n \n \tsource = odb_find_source(the_repository->objects, opts.obj_dir);\n+\tif (!source)\n+\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n \n \tif (opts.reachable) {\n \t\tif (write_commit_graph_reachable(source, flags, &write_opts))\ndiff --git a/midx-write.c b/midx-write.c\nindex c1ae62d354..40580d8c73 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -917,6 +917,8 @@ static struct multi_pack_index *lookup_multi_pack_index(struct repository *r,\n \t\t\t\t\t\t\tconst char *object_dir)\n {\n \tstruct odb_source *source = odb_find_source(r->objects, object_dir);\n+\tif (!source)\n+\t\tdie(_(\"could not find object directory matching %s\"), object_dir);\n \treturn get_multi_pack_index(source);\n }\n \ndiff --git a/odb.c b/odb.c\nindex 1761a50840..61104b7cb8 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -464,8 +464,6 @@ struct odb_source *odb_find_source(struct object_database *odb, const char *obj_\n \tfree(obj_dir_real);\n \tstrbuf_release(&odb_path_real);\n \n-\tif (!source)\n-\t\tdie(_(\"could not find object directory matching %s\"), obj_dir);\n \treturn source;\n }\n \ndiff --git a/odb.h b/odb.h\nindex d9f4dcf79f..387b117c87 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -186,8 +186,8 @@ struct object_database *odb_new(struct repository *repo);\n void odb_clear(struct object_database *o);\n \n /*\n- * Find source by its object directory path. Dies in case the source couldn't\n- * be found.\n+ * Find source by its object directory path. Returns a `NULL` pointer in case\n+ * the source could not be found.\n */\n struct odb_source *odb_find_source(struct object_database *odb, const char *obj_dir);\n \n\n-- \n2.50.1.619.g074bbf1d35.dirty\n\n"},{"id":"522956","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-3-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH 3/8] odb: return newly created in-memory sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:42Z","receivedAt":"2025-07-29T14:12:55Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Callers have no trivial way to obtain the newly created object database\nsource when adding it to the in-memory list of alternates. While not yet\nneeded anywhere, a subsequent commit will want to obtain that pointer.\n\nRefactor the function to return the source to make it easily accessible.\n\nSigned-off-by: Patrick Steinhardt \n---\n odb.c | 37 +++++++++++++++++++++----------------\n odb.h | 4 ++--\n 2 files changed, 23 insertions(+), 18 deletions(-)\n\ndiff --git a/odb.c b/odb.c\nindex 61104b7cb8..7793816f81 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -139,23 +139,22 @@ static void read_info_alternates(struct object_database *odb,\n \t\t\t\t const char *relative_base,\n \t\t\t\t int depth);\n \n-static int link_alt_odb_entry(struct object_database *odb,\n-\t\t\t const struct strbuf *entry,\n-\t\t\t const char *relative_base,\n-\t\t\t int depth,\n-\t\t\t const char *normalized_objdir)\n+static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n+\t\t\t\t\t const char *entry,\n+\t\t\t\t\t const char *relative_base,\n+\t\t\t\t\t int depth,\n+\t\t\t\t\t const char *normalized_objdir)\n {\n-\tstruct odb_source *alternate;\n+\tstruct odb_source *alternate = NULL;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n \tstruct strbuf tmp = STRBUF_INIT;\n \tkhiter_t pos;\n-\tint ret = -1;\n \n-\tif (!is_absolute_path(entry->buf) && relative_base) {\n+\tif (!is_absolute_path(entry) && relative_base) {\n \t\tstrbuf_realpath(&pathbuf, relative_base, 1);\n \t\tstrbuf_addch(&pathbuf, '/');\n \t}\n-\tstrbuf_addbuf(&pathbuf, entry);\n+\tstrbuf_addstr(&pathbuf, entry);\n \n \tif (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {\n \t\terror(_(\"unable to normalize alternate object path: %s\"),\n@@ -189,11 +188,11 @@ static int link_alt_odb_entry(struct object_database *odb,\n \n \t/* recursively add alternates */\n \tread_info_alternates(odb, alternate->path, depth + 1);\n-\tret = 0;\n+\n error:\n \tstrbuf_release(&tmp);\n \tstrbuf_release(&pathbuf);\n-\treturn ret;\n+\treturn alternate;\n }\n \n static const char *parse_alt_odb_entry(const char *string,\n@@ -246,7 +245,7 @@ static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(odb, &entry,\n+\t\tlink_alt_odb_entry(odb, entry.buf,\n \t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n@@ -316,17 +315,23 @@ void odb_add_to_alternates_file(struct object_database *odb,\n \tfree(alts);\n }\n \n-void odb_add_to_alternates_memory(struct object_database *odb,\n-\t\t\t\t const char *reference)\n+struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n+\t\t\t\t\t\tconst char *reference)\n {\n+\tstruct odb_source *alternate;\n+\tchar *objdir;\n+\n \t/*\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n \todb_prepare_alternates(odb);\n \n-\tlink_alt_odb_entries(odb, reference,\n-\t\t\t '\\n', NULL, 0);\n+\tobjdir = real_pathdup(odb->sources->path, 1);\n+\talternate = link_alt_odb_entry(odb, reference, NULL, 0, objdir);\n+\n+\tfree(objdir);\n+\treturn alternate;\n }\n \n struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,\ndiff --git a/odb.h b/odb.h\nindex 387b117c87..ac7ee05188 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -265,8 +265,8 @@ void odb_add_to_alternates_file(struct object_database *odb,\n * recursive alternates it points to), but do not modify the on-disk alternates\n * file.\n */\n-void odb_add_to_alternates_memory(struct object_database *odb,\n-\t\t\t\t const char *dir);\n+struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n+\t\t\t\t\t\tconst char *dir);\n \n /*\n * Read an object from the database. Returns the object data and assigns object\n\n-- \n2.50.1.619.g074bbf1d35.dirty\n\n"},{"id":"522957","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-4-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH 4/8] midx: drop redundant `struct repository` parameter","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:43Z","receivedAt":"2025-07-29T14:12:57Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"There are a couple of functions that take both a `struct repository` and\na `struct multi_pack_index`. This provides redundant information though\nwithout much benefit given that the multi-pack index already has a\npointer to its owning repository.\n\nDrop the `struct repository` parameter from such functions. While at it,\nreorder the list of parameters of `fill_midx_entry()` so that the MIDX\ncomes first to better align with our coding guidelines.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/pack-objects.c | 2 +-\n midx-write.c | 16 +++++++---------\n midx.c | 18 +++++++++---------\n midx.h | 6 +++---\n pack-bitmap.c | 4 ++--\n packfile.c | 4 ++--\n t/helper/test-read-midx.c | 4 ++--\n 7 files changed, 26 insertions(+), 28 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 3dd84495b8..b9fd685b8f 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1733,7 +1733,7 @@ static int want_object_in_pack_mtime(const struct object_id *oid,\n \t\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \t\tstruct pack_entry e;\n \n-\t\tif (m && fill_midx_entry(the_repository, oid, &e, m)) {\n+\t\tif (m && fill_midx_entry(m, oid, &e)) {\n \t\t\twant = want_object_in_pack_one(e.p, oid, exclude, found_pack, found_offset, found_mtime);\n \t\t\tif (want != -1)\n \t\t\t\treturn want;\ndiff --git a/midx-write.c b/midx-write.c\nindex 40580d8c73..37a0b1682f 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -944,8 +944,7 @@ static int fill_packs_from_midx(struct write_midx_context *ctx,\n \t\t\t */\n \t\t\tif (flags & MIDX_WRITE_REV_INDEX ||\n \t\t\t preferred_pack_name) {\n-\t\t\t\tif (prepare_midx_pack(ctx->repo, m,\n-\t\t\t\t\t\t m->num_packs_in_base + i)) {\n+\t\t\t\tif (prepare_midx_pack(m, m->num_packs_in_base + i)) {\n \t\t\t\t\terror(_(\"could not load pack\"));\n \t\t\t\t\treturn 1;\n \t\t\t\t}\n@@ -1568,7 +1567,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \t\tif (count[i])\n \t\t\tcontinue;\n \n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tcontinue;\n \n \t\tif (m->packs[i]->pack_keep || m->packs[i]->is_cruft)\n@@ -1614,13 +1613,12 @@ static int compare_by_mtime(const void *a_, const void *b_)\n \treturn 0;\n }\n \n-static int want_included_pack(struct repository *r,\n-\t\t\t struct multi_pack_index *m,\n+static int want_included_pack(struct multi_pack_index *m,\n \t\t\t int pack_kept_objects,\n \t\t\t uint32_t pack_int_id)\n {\n \tstruct packed_git *p;\n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn 0;\n \tp = m->packs[pack_int_id];\n \tif (!pack_kept_objects && p->pack_keep)\n@@ -1642,7 +1640,7 @@ static void fill_included_packs_all(struct repository *r,\n \trepo_config_get_bool(r, \"repack.packkeptobjects\", &pack_kept_objects);\n \n \tfor (i = 0; i < m->num_packs; i++) {\n-\t\tif (!want_included_pack(r, m, pack_kept_objects, i))\n+\t\tif (!want_included_pack(m, pack_kept_objects, i))\n \t\t\tcontinue;\n \n \t\tinclude_pack[i] = 1;\n@@ -1666,7 +1664,7 @@ static void fill_included_packs_batch(struct repository *r,\n \tfor (i = 0; i < m->num_packs; i++) {\n \t\tpack_info[i].pack_int_id = i;\n \n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tcontinue;\n \n \t\tpack_info[i].mtime = m->packs[i]->mtime;\n@@ -1685,7 +1683,7 @@ static void fill_included_packs_batch(struct repository *r,\n \t\tstruct packed_git *p = m->packs[pack_int_id];\n \t\tuint64_t expected_size;\n \n-\t\tif (!want_included_pack(r, m, pack_kept_objects, pack_int_id))\n+\t\tif (!want_included_pack(m, pack_kept_objects, pack_int_id))\n \t\t\tcontinue;\n \n \t\t/*\ndiff --git a/midx.c b/midx.c\nindex b9ca0915a6..8459dda8c9 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -450,9 +450,10 @@ static uint32_t midx_for_pack(struct multi_pack_index **_m,\n \treturn pack_int_id - m->num_packs_in_base;\n }\n \n-int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,\n+int prepare_midx_pack(struct multi_pack_index *m,\n \t\t uint32_t pack_int_id)\n {\n+\tstruct repository *r = m->repo;\n \tstruct strbuf pack_name = STRBUF_INIT;\n \tstruct strbuf key = STRBUF_INIT;\n \tstruct packed_git *p;\n@@ -507,7 +508,7 @@ struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \n #define MIDX_CHUNK_BITMAPPED_PACKS_WIDTH (2 * sizeof(uint32_t))\n \n-int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n+int nth_bitmapped_pack(struct multi_pack_index *m,\n \t\t struct bitmapped_pack *bp, uint32_t pack_int_id)\n {\n \tuint32_t local_pack_int_id = midx_for_pack(&m, pack_int_id);\n@@ -515,7 +516,7 @@ int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n \tif (!m->chunk_bitmapped_packs)\n \t\treturn error(_(\"MIDX does not contain the BTMP chunk\"));\n \n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn error(_(\"could not load bitmapped pack %\"PRIu32), pack_int_id);\n \n \tbp->p = m->packs[local_pack_int_id];\n@@ -600,10 +601,9 @@ uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos)\n \t\t\t\t\t (off_t)pos * MIDX_CHUNK_OFFSET_WIDTH);\n }\n \n-int fill_midx_entry(struct repository *r,\n+int fill_midx_entry(struct multi_pack_index *m,\n \t\t const struct object_id *oid,\n-\t\t struct pack_entry *e,\n-\t\t struct multi_pack_index *m)\n+\t\t struct pack_entry *e)\n {\n \tuint32_t pos;\n \tuint32_t pack_int_id;\n@@ -615,7 +615,7 @@ int fill_midx_entry(struct repository *r,\n \tmidx_for_object(&m, pos);\n \tpack_int_id = nth_midxed_pack_int_id(m, pos);\n \n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn 0;\n \tp = m->packs[pack_int_id - m->num_packs_in_base];\n \n@@ -912,7 +912,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \t\t\t\t\t\t _(\"Looking for referenced packfiles\"),\n \t\t\t\t\t\t m->num_packs + m->num_packs_in_base);\n \tfor (i = 0; i < m->num_packs + m->num_packs_in_base; i++) {\n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tmidx_report(\"failed to load pack in position %d\", i);\n \n \t\tdisplay_progress(progress, i + 1);\n@@ -989,7 +989,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \n \t\tnth_midxed_object_oid(&oid, m, pairs[i].pos);\n \n-\t\tif (!fill_midx_entry(r, &oid, &e, m)) {\n+\t\tif (!fill_midx_entry(m, &oid, &e)) {\n \t\t\tmidx_report(_(\"failed to load pack entry for oid[%d] = %s\"),\n \t\t\t\t pairs[i].pos, oid_to_hex(&oid));\n \t\t\tcontinue;\ndiff --git a/midx.h b/midx.h\nindex 28c426a823..f7e07083e1 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -103,10 +103,10 @@ void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n struct multi_pack_index *load_multi_pack_index(struct repository *r,\n \t\t\t\t\t const char *object_dir,\n \t\t\t\t\t int local);\n-int prepare_midx_pack(struct repository *r, struct multi_pack_index *m, uint32_t pack_int_id);\n+int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);\n struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \t\t\t\t uint32_t pack_int_id);\n-int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n+int nth_bitmapped_pack(struct multi_pack_index *m,\n \t\t struct bitmapped_pack *bp, uint32_t pack_int_id);\n int bsearch_one_midx(const struct object_id *oid, struct multi_pack_index *m,\n \t\t uint32_t *result);\n@@ -118,7 +118,7 @@ uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos);\n struct object_id *nth_midxed_object_oid(struct object_id *oid,\n \t\t\t\t\tstruct multi_pack_index *m,\n \t\t\t\t\tuint32_t n);\n-int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pack_entry *e, struct multi_pack_index *m);\n+int fill_midx_entry(struct multi_pack_index *m, const struct object_id *oid, struct pack_entry *e);\n int midx_contains_pack(struct multi_pack_index *m,\n \t\t const char *idx_or_pack_name);\n int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex d14421ee20..fb0b11ca07 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -493,7 +493,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t}\n \n \tfor (i = 0; i < bitmap_git->midx->num_packs + bitmap_git->midx->num_packs_in_base; i++) {\n-\t\tif (prepare_midx_pack(bitmap_repo(bitmap_git), bitmap_git->midx, i)) {\n+\t\tif (prepare_midx_pack(bitmap_git->midx, i)) {\n \t\t\twarning(_(\"could not open pack %s\"),\n \t\t\t\tbitmap_git->midx->pack_names[i]);\n \t\t\tgoto cleanup;\n@@ -2466,7 +2466,7 @@ void reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git,\n \t\tstruct multi_pack_index *m = bitmap_git->midx;\n \t\tfor (i = 0; i < m->num_packs + m->num_packs_in_base; i++) {\n \t\t\tstruct bitmapped_pack pack;\n-\t\t\tif (nth_bitmapped_pack(r, bitmap_git->midx, &pack, i) < 0) {\n+\t\t\tif (nth_bitmapped_pack(bitmap_git->midx, &pack, i) < 0) {\n \t\t\t\twarning(_(\"unable to load pack: '%s', disabling pack-reuse\"),\n \t\t\t\t\tbitmap_git->midx->pack_names[i]);\n \t\t\t\tfree(packs);\ndiff --git a/packfile.c b/packfile.c\nindex a38544b87b..acb680966d 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -1091,7 +1091,7 @@ struct packed_git *get_all_packs(struct repository *r)\n \t\tif (!m)\n \t\t\tcontinue;\n \t\tfor (uint32_t i = 0; i < m->num_packs + m->num_packs_in_base; i++)\n-\t\t\tprepare_midx_pack(r, m, i);\n+\t\t\tprepare_midx_pack(m, i);\n \t}\n \n \treturn r->objects->packed_git;\n@@ -2077,7 +2077,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa\n \tprepare_packed_git(r);\n \n \tfor (struct odb_source *source = r->objects->sources; source; source = source->next)\n-\t\tif (source->midx && fill_midx_entry(r, oid, e, source->midx))\n+\t\tif (source->midx && fill_midx_entry(source->midx, oid, e))\n \t\t\treturn 1;\n \n \tif (!r->objects->packed_git)\ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex da2aa036b5..e430aa247c 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -65,7 +65,7 @@ static int read_midx_file(const char *object_dir, const char *checksum,\n \t\tfor (i = 0; i < m->num_objects; i++) {\n \t\t\tnth_midxed_object_oid(&oid, m,\n \t\t\t\t\t i + m->num_objects_in_base);\n-\t\t\tfill_midx_entry(the_repository, &oid, &e, m);\n+\t\t\tfill_midx_entry(m, &oid, &e);\n \n \t\t\tprintf(\"%s %\"PRIu64\"\\t%s\\n\",\n \t\t\t oid_to_hex(&oid), e.offset, e.p->pack_name);\n@@ -126,7 +126,7 @@ static int read_midx_bitmapped_packs(const char *object_dir)\n \t\treturn 1;\n \n \tfor (i = 0; i < midx->num_packs + midx->num_packs_in_base; i++) {\n-\t\tif (nth_bitmapped_pack(the_repository, midx, &pack, i) < 0) {\n+\t\tif (nth_bitmapped_pack(midx, &pack, i) < 0) {\n \t\t\tclose_midx(midx);\n \t\t\treturn 1;\n \t\t}\n\n-- \n2.50.1.619.g074bbf1d35.dirty\n\n"},{"id":"522958","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-5-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH 5/8] midx: load multi-pack indices via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:44Z","receivedAt":"2025-07-29T14:13:01Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"To load a multi-pack index the caller is expected to pass both the\nrepository and the object directory where the multi-pack index is\nlocated. While this works, this layout has a couple of downsides:\n\n - We need to pass in information reduntant with the owning source,\n namely its object directory and whether the source is local or not.\n\n - We don't have access to the source when loading the multi-pack\n index. If we had that access, we could store a pointer to the owning\n source in the MIDX and thus deduplicate some information.\n\n - Multi-pack indices are inherently specific to the object source and\n its format. With the goal of pluggable object backends in mind we\n will eventually want the backends to own the logic of reading and\n writing multi-pack indices. Making the logic work on top of object\n sources is a step into that direction.\n\nRefactor loading of multi-pack indices accordingly.\n\nThis surfaces one small problem though: git-multi-pack-index(1) and our\nMIDX test helper both know to read and write multi-pack-indices located\nin a different object directory. This issue is addressed by adding the\nuser-provided object directory as an in-memory alternate.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/multi-pack-index.c | 18 ++++++++++++--\n midx.c | 57 ++++++++++++++++++++-------------------------\n midx.h | 6 ++---\n t/helper/test-read-midx.c | 25 ++++++++++++--------\n t/t5319-multi-pack-index.sh | 8 +++----\n 5 files changed, 62 insertions(+), 52 deletions(-)\n\ndiff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c\nindex aa25b06f9d..e4a9305af3 100644\n--- a/builtin/multi-pack-index.c\n+++ b/builtin/multi-pack-index.c\n@@ -64,12 +64,20 @@ static int parse_object_dir(const struct option *opt, const char *arg,\n \tchar **value = opt->value;\n \tfree(*value);\n \tif (unset)\n-\t\t*value = xstrdup(repo_get_object_directory(the_repository));\n+\t\t*value = xstrdup(the_repository->objects->sources->path);\n \telse\n \t\t*value = real_pathdup(arg, 1);\n \treturn 0;\n }\n \n+static struct odb_source *handle_object_dir_option(struct repository *repo)\n+{\n+\tstruct odb_source *source = odb_find_source(repo->objects, opts.object_dir);\n+\tif (!source)\n+\t\tsource = odb_add_to_alternates_memory(repo->objects, opts.object_dir);\n+\treturn source;\n+}\n+\n static struct option common_opts[] = {\n \tOPT_CALLBACK(0, \"object-dir\", &opts.object_dir,\n \t N_(\"directory\"),\n@@ -157,6 +165,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_write_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(repo);\n \n \tFREE_AND_NULL(options);\n \n@@ -193,6 +202,8 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,\n \t\t\tN_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n+\n \toptions = add_common_options(builtin_multi_pack_index_verify_options);\n \n \ttrace2_cmd_mode(argv[0]);\n@@ -205,10 +216,11 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_verify_usage,\n \t\t\t\t options);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn verify_midx_file(the_repository, opts.object_dir, opts.flags);\n+\treturn verify_midx_file(source, opts.flags);\n }\n \n static int cmd_multi_pack_index_expire(int argc, const char **argv,\n@@ -233,6 +245,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_expire_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n@@ -265,6 +278,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_repack_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \ndiff --git a/midx.c b/midx.c\nindex 8459dda8c9..831a7e9b5f 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -95,11 +95,10 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,\n \treturn 0;\n }\n \n-static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,\n-\t\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t\t const char *midx_name,\n-\t\t\t\t\t\t\t int local)\n+static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,\n+\t\t\t\t\t\t\t const char *midx_name)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct multi_pack_index *m = NULL;\n \tint fd;\n \tstruct stat st;\n@@ -129,10 +128,10 @@ static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,\n \tmidx_map = xmmap(NULL, midx_size, PROT_READ, MAP_PRIVATE, fd, 0);\n \tclose(fd);\n \n-\tFLEX_ALLOC_STR(m, object_dir, object_dir);\n+\tFLEX_ALLOC_STR(m, object_dir, source->path);\n \tm->data = midx_map;\n \tm->data_len = midx_size;\n-\tm->local = local;\n+\tm->local = source->local;\n \tm->repo = r;\n \n \tm->signature = get_be32(m->data);\n@@ -297,19 +296,18 @@ static int add_midx_to_chain(struct multi_pack_index *midx,\n \treturn 1;\n }\n \n-static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n-\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t int local,\n+static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source *source,\n \t\t\t\t\t\t int fd, struct stat *st,\n \t\t\t\t\t\t int *incomplete_chain)\n {\n+\tconst struct git_hash_algo *hash_algo = source->odb->repo->hash_algo;\n \tstruct multi_pack_index *midx_chain = NULL;\n \tstruct strbuf buf = STRBUF_INIT;\n \tint valid = 1;\n \tuint32_t i, count;\n \tFILE *fp = xfdopen(fd, \"r\");\n \n-\tcount = st->st_size / (r->hash_algo->hexsz + 1);\n+\tcount = st->st_size / (hash_algo->hexsz + 1);\n \n \tfor (i = 0; i < count; i++) {\n \t\tstruct multi_pack_index *m;\n@@ -318,7 +316,7 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \t\tif (strbuf_getline_lf(&buf, fp) == EOF)\n \t\t\tbreak;\n \n-\t\tif (get_oid_hex_algop(buf.buf, &layer, r->hash_algo)) {\n+\t\tif (get_oid_hex_algop(buf.buf, &layer, hash_algo)) {\n \t\t\twarning(_(\"invalid multi-pack-index chain: line '%s' \"\n \t\t\t\t \"not a hash\"),\n \t\t\t\tbuf.buf);\n@@ -329,9 +327,9 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \t\tvalid = 0;\n \n \t\tstrbuf_reset(&buf);\n-\t\tget_split_midx_filename_ext(r->hash_algo, &buf, object_dir,\n+\t\tget_split_midx_filename_ext(hash_algo, &buf, source->path,\n \t\t\t\t\t layer.hash, MIDX_EXT_MIDX);\n-\t\tm = load_multi_pack_index_one(r, object_dir, buf.buf, local);\n+\t\tm = load_multi_pack_index_one(source, buf.buf);\n \n \t\tif (m) {\n \t\t\tif (add_midx_to_chain(m, midx_chain)) {\n@@ -354,40 +352,35 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \treturn midx_chain;\n }\n \n-static struct multi_pack_index *load_multi_pack_index_chain(struct repository *r,\n-\t\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t\t int local)\n+static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *source)\n {\n \tstruct strbuf chain_file = STRBUF_INIT;\n \tstruct stat st;\n \tint fd;\n \tstruct multi_pack_index *m = NULL;\n \n-\tget_midx_chain_filename(&chain_file, object_dir);\n-\tif (open_multi_pack_index_chain(r->hash_algo, chain_file.buf, &fd, &st)) {\n+\tget_midx_chain_filename(&chain_file, source->path);\n+\tif (open_multi_pack_index_chain(source->odb->repo->hash_algo, chain_file.buf, &fd, &st)) {\n \t\tint incomplete;\n \t\t/* ownership of fd is taken over by load function */\n-\t\tm = load_midx_chain_fd_st(r, object_dir, local, fd, &st,\n-\t\t\t\t\t &incomplete);\n+\t\tm = load_midx_chain_fd_st(source, fd, &st, &incomplete);\n \t}\n \n \tstrbuf_release(&chain_file);\n \treturn m;\n }\n \n-struct multi_pack_index *load_multi_pack_index(struct repository *r,\n-\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t int local)\n+struct multi_pack_index *load_multi_pack_index(struct odb_source *source)\n {\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tstruct multi_pack_index *m;\n \n-\tget_midx_filename(r->hash_algo, &midx_name, object_dir);\n+\tget_midx_filename(source->odb->repo->hash_algo, &midx_name,\n+\t\t\t source->path);\n \n-\tm = load_multi_pack_index_one(r, object_dir,\n-\t\t\t\t midx_name.buf, local);\n+\tm = load_multi_pack_index_one(source, midx_name.buf);\n \tif (!m)\n-\t\tm = load_multi_pack_index_chain(r, object_dir, local);\n+\t\tm = load_multi_pack_index_chain(source);\n \n \tstrbuf_release(&midx_name);\n \n@@ -734,8 +727,7 @@ int prepare_multi_pack_index_one(struct odb_source *source)\n \tif (source->midx)\n \t\treturn 1;\n \n-\tsource->midx = load_multi_pack_index(r, source->path,\n-\t\t\t\t\t source->local);\n+\tsource->midx = load_multi_pack_index(source);\n \n \treturn !!source->midx;\n }\n@@ -880,12 +872,13 @@ static int compare_pair_pos_vs_id(const void *_a, const void *_b)\n \t\t\tdisplay_progress(progress, _n); \\\n \t} while (0)\n \n-int verify_midx_file(struct repository *r, const char *object_dir, unsigned flags)\n+int verify_midx_file(struct odb_source *source, unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct pair_pos_vs_id *pairs = NULL;\n \tuint32_t i;\n \tstruct progress *progress = NULL;\n-\tstruct multi_pack_index *m = load_multi_pack_index(r, object_dir, 1);\n+\tstruct multi_pack_index *m = load_multi_pack_index(source);\n \tstruct multi_pack_index *curr;\n \tverify_midx_error = 0;\n \n@@ -894,7 +887,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \t\tstruct stat sb;\n \t\tstruct strbuf filename = STRBUF_INIT;\n \n-\t\tget_midx_filename(r->hash_algo, &filename, object_dir);\n+\t\tget_midx_filename(r->hash_algo, &filename, source->path);\n \n \t\tif (!stat(filename.buf, &sb)) {\n \t\t\terror(_(\"multi-pack-index file exists, but failed to parse\"));\ndiff --git a/midx.h b/midx.h\nindex f7e07083e1..970d043989 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -100,9 +100,7 @@ void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n \t\t\t\t struct strbuf *buf, const char *object_dir,\n \t\t\t\t const unsigned char *hash, const char *ext);\n \n-struct multi_pack_index *load_multi_pack_index(struct repository *r,\n-\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t int local);\n+struct multi_pack_index *load_multi_pack_index(struct odb_source *source);\n int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);\n struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \t\t\t\t uint32_t pack_int_id);\n@@ -136,7 +134,7 @@ int write_midx_file_only(struct repository *r, const char *object_dir,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags);\n void clear_midx_file(struct repository *r);\n-int verify_midx_file(struct repository *r, const char *object_dir, unsigned flags);\n+int verify_midx_file(struct odb_source *source, unsigned flags);\n int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags);\n int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags);\n \ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex e430aa247c..bcb8ea7671 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -11,14 +11,24 @@\n #include \"gettext.h\"\n #include \"pack-revindex.h\"\n \n+static struct multi_pack_index *setup_midx(const char *object_dir)\n+{\n+\tstruct odb_source *source;\n+\tsetup_git_directory();\n+\tsource = odb_find_source(the_repository->objects, object_dir);\n+\tif (!source)\n+\t\tsource = odb_add_to_alternates_memory(the_repository->objects,\n+\t\t\t\t\t\t object_dir);\n+\treturn load_multi_pack_index(source);\n+}\n+\n static int read_midx_file(const char *object_dir, const char *checksum,\n \t\t\t int show_objects)\n {\n \tuint32_t i;\n \tstruct multi_pack_index *m;\n \n-\tsetup_git_directory();\n-\tm = load_multi_pack_index(the_repository, object_dir, 1);\n+\tm = setup_midx(object_dir);\n \n \tif (!m)\n \t\treturn 1;\n@@ -81,8 +91,7 @@ static int read_midx_checksum(const char *object_dir)\n {\n \tstruct multi_pack_index *m;\n \n-\tsetup_git_directory();\n-\tm = load_multi_pack_index(the_repository, object_dir, 1);\n+\tm = setup_midx(object_dir);\n \tif (!m)\n \t\treturn 1;\n \tprintf(\"%s\\n\", hash_to_hex(get_midx_checksum(m)));\n@@ -96,9 +105,7 @@ static int read_midx_preferred_pack(const char *object_dir)\n \tstruct multi_pack_index *midx = NULL;\n \tuint32_t preferred_pack;\n \n-\tsetup_git_directory();\n-\n-\tmidx = load_multi_pack_index(the_repository, object_dir, 1);\n+\tmidx = setup_midx(object_dir);\n \tif (!midx)\n \t\treturn 1;\n \n@@ -119,9 +126,7 @@ static int read_midx_bitmapped_packs(const char *object_dir)\n \tstruct bitmapped_pack pack;\n \tuint32_t i;\n \n-\tsetup_git_directory();\n-\n-\tmidx = load_multi_pack_index(the_repository, object_dir, 1);\n+\tmidx = setup_midx(object_dir);\n \tif (!midx)\n \t\treturn 1;\n \ndiff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh\nindex bd75dea950..4e5e882989 100755\n--- a/t/t5319-multi-pack-index.sh\n+++ b/t/t5319-multi-pack-index.sh\n@@ -28,11 +28,11 @@ midx_read_expect () {\n \t\tEOF\n \t\tif test $NUM_PACKS -ge 1\n \t\tthen\n-\t\t\tls $OBJECT_DIR/pack/ | grep idx | sort\n+\t\t\tls \"$OBJECT_DIR\"/pack/ | grep idx | sort\n \t\tfi &&\n \t\tprintf \"object-dir: $OBJECT_DIR\\n\"\n \t} >expect &&\n-\ttest-tool read-midx $OBJECT_DIR >actual &&\n+\ttest-tool read-midx \"$OBJECT_DIR\" >actual &&\n \ttest_cmp expect actual\n }\n \n@@ -305,7 +305,7 @@ test_expect_success 'midx picks objects from preferred pack' '\n \n \t\tofs=$(git show-index expect &&\n \t\tgrep ^$b out >actual &&\n \n@@ -639,7 +639,7 @@ test_expect_success 'force some 64-bit offsets with pack-objects' '\n \t\t( cd ../objects64 && pwd ) >.git/objects/info/alternates &&\n \t\tmidx64=$(git multi-pack-index --object-dir=../objects64 write)\n \t) &&\n-\tmidx_read_expect 1 63 5 objects64 \" large-offsets\"\n+\tmidx_read_expect 1 63 5 \"$(pwd)/objects64\" \" large-offsets\"\n '\n \n test_expect_success 'verify multi-pack-index with 64-bit offsets' '\n\n-- \n2.50.1.619.g074bbf1d35.dirty\n\n"},{"id":"522959","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-6-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH 6/8] midx: write multi-pack indices via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:45Z","receivedAt":"2025-07-29T14:13:03Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Similar to the preceding commit, refactor the writing side of multi-pack\nindices so that we pass in the object database source where the index\nshould be written to.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/multi-pack-index.c | 19 +++++++------\n builtin/repack.c | 2 +-\n midx-write.c | 69 +++++++++++++++++++++-------------------------\n midx.h | 8 +++---\n 4 files changed, 47 insertions(+), 51 deletions(-)\n\ndiff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c\nindex e4a9305af3..b1e971e535 100644\n--- a/builtin/multi-pack-index.c\n+++ b/builtin/multi-pack-index.c\n@@ -147,6 +147,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \t\t\t N_(\"refs snapshot for selecting bitmap commits\")),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n \tint ret;\n \n \topts.flags |= MIDX_WRITE_BITMAP_HASH_CACHE;\n@@ -165,7 +166,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_write_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(repo);\n+\tsource = handle_object_dir_option(repo);\n \n \tFREE_AND_NULL(options);\n \n@@ -174,7 +175,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \n \t\tread_packs_from_stdin(&packs);\n \n-\t\tret = write_midx_file_only(repo, opts.object_dir, &packs,\n+\t\tret = write_midx_file_only(source, &packs,\n \t\t\t\t\t opts.preferred_pack,\n \t\t\t\t\t opts.refs_snapshot, opts.flags);\n \n@@ -185,7 +186,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \n \t}\n \n-\tret = write_midx_file(repo, opts.object_dir, opts.preferred_pack,\n+\tret = write_midx_file(source, opts.preferred_pack,\n \t\t\t opts.refs_snapshot, opts.flags);\n \n \tfree(opts.refs_snapshot);\n@@ -233,6 +234,8 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \t\t\tN_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n+\n \toptions = add_common_options(builtin_multi_pack_index_expire_options);\n \n \ttrace2_cmd_mode(argv[0]);\n@@ -245,11 +248,11 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_expire_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(the_repository);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn expire_midx_packs(the_repository, opts.object_dir, opts.flags);\n+\treturn expire_midx_packs(source, opts.flags);\n }\n \n static int cmd_multi_pack_index_repack(int argc, const char **argv,\n@@ -264,6 +267,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \t\t N_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n \n \toptions = add_common_options(builtin_multi_pack_index_repack_options);\n \n@@ -278,12 +282,11 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_repack_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(the_repository);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn midx_repack(the_repository, opts.object_dir,\n-\t\t\t (size_t)opts.batch_size, opts.flags);\n+\treturn midx_repack(source, (size_t)opts.batch_size, opts.flags);\n }\n \n int cmd_multi_pack_index(int argc,\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 21723866b9..94dec26f18 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -1711,7 +1711,7 @@ int cmd_repack(int argc,\n \t\tunsigned flags = 0;\n \t\tif (git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL, 0))\n \t\t\tflags |= MIDX_WRITE_INCREMENTAL;\n-\t\twrite_midx_file(the_repository, repo_get_object_directory(the_repository),\n+\t\twrite_midx_file(the_repository->objects->sources,\n \t\t\t\tNULL, NULL, flags);\n \t}\n \ndiff --git a/midx-write.c b/midx-write.c\nindex 37a0b1682f..bf7c01d4b1 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -913,15 +913,6 @@ static int write_midx_bitmap(struct write_midx_context *ctx,\n \treturn ret;\n }\n \n-static struct multi_pack_index *lookup_multi_pack_index(struct repository *r,\n-\t\t\t\t\t\t\tconst char *object_dir)\n-{\n-\tstruct odb_source *source = odb_find_source(r->objects, object_dir);\n-\tif (!source)\n-\t\tdie(_(\"could not find object directory matching %s\"), object_dir);\n-\treturn get_multi_pack_index(source);\n-}\n-\n static int fill_packs_from_midx(struct write_midx_context *ctx,\n \t\t\t\tconst char *preferred_pack_name, uint32_t flags)\n {\n@@ -1012,7 +1003,7 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \treturn ret;\n }\n \n-static void clear_midx_files(struct repository *r, const char *object_dir,\n+static void clear_midx_files(struct odb_source *source,\n \t\t\t const char **hashes, uint32_t hashes_nr,\n \t\t\t unsigned incremental)\n {\n@@ -1031,16 +1022,16 @@ static void clear_midx_files(struct repository *r, const char *object_dir,\n \tuint32_t i, j;\n \n \tfor (i = 0; i < ARRAY_SIZE(exts); i++) {\n-\t\tclear_incremental_midx_files_ext(object_dir, exts[i],\n+\t\tclear_incremental_midx_files_ext(source->path, exts[i],\n \t\t\t\t\t\t hashes, hashes_nr);\n \t\tfor (j = 0; j < hashes_nr; j++)\n-\t\t\tclear_midx_files_ext(object_dir, exts[i], hashes[j]);\n+\t\t\tclear_midx_files_ext(source->path, exts[i], hashes[j]);\n \t}\n \n \tif (incremental)\n-\t\tget_midx_filename(r->hash_algo, &buf, object_dir);\n+\t\tget_midx_filename(source->odb->repo->hash_algo, &buf, source->path);\n \telse\n-\t\tget_midx_chain_filename(&buf, object_dir);\n+\t\tget_midx_chain_filename(&buf, source->path);\n \n \tif (unlink(buf.buf) && errno != ENOENT)\n \t\tdie_errno(_(\"failed to clear multi-pack-index at %s\"), buf.buf);\n@@ -1048,13 +1039,14 @@ static void clear_midx_files(struct repository *r, const char *object_dir,\n \tstrbuf_release(&buf);\n }\n \n-static int write_midx_internal(struct repository *r, const char *object_dir,\n+static int write_midx_internal(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t struct string_list *packs_to_drop,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot,\n \t\t\t unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tunsigned char midx_hash[GIT_MAX_RAWSZ];\n \tuint32_t i, start_pack;\n@@ -1078,15 +1070,15 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (ctx.incremental)\n \t\tstrbuf_addf(&midx_name,\n \t\t\t \"%s/pack/multi-pack-index.d/tmp_midx_XXXXXX\",\n-\t\t\t object_dir);\n+\t\t\t source->path);\n \telse\n-\t\tget_midx_filename(r->hash_algo, &midx_name, object_dir);\n+\t\tget_midx_filename(r->hash_algo, &midx_name, source->path);\n \tif (safe_create_leading_directories(r, midx_name.buf))\n \t\tdie_errno(_(\"unable to create leading directories of %s\"),\n \t\t\t midx_name.buf);\n \n \tif (!packs_to_include || ctx.incremental) {\n-\t\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\t\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \t\tif (m && !midx_checksum_valid(m)) {\n \t\t\twarning(_(\"ignoring existing multi-pack-index; checksum mismatch\"));\n \t\t\tm = NULL;\n@@ -1140,7 +1132,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \n \tctx.to_include = packs_to_include;\n \n-\tfor_each_file_in_pack_dir(object_dir, add_pack_to_midx, &ctx);\n+\tfor_each_file_in_pack_dir(source->path, add_pack_to_midx, &ctx);\n \tstop_progress(&ctx.progress);\n \n \tif ((ctx.m && ctx.nr == ctx.m->num_packs + ctx.m->num_packs_in_base) &&\n@@ -1160,7 +1152,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\t\t * corresponding bitmap (or one wasn't requested).\n \t\t\t */\n \t\t\tif (!want_bitmap)\n-\t\t\t\tclear_midx_files_ext(object_dir, \"bitmap\", NULL);\n+\t\t\t\tclear_midx_files_ext(source->path, \"bitmap\", NULL);\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -1328,7 +1320,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (ctx.incremental) {\n \t\tstruct strbuf lock_name = STRBUF_INIT;\n \n-\t\tget_midx_chain_filename(&lock_name, object_dir);\n+\t\tget_midx_chain_filename(&lock_name, source->path);\n \t\thold_lock_file_for_update(&lk, lock_name.buf, LOCK_DIE_ON_ERROR);\n \t\tstrbuf_release(&lock_name);\n \n@@ -1391,7 +1383,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \n \tif (flags & MIDX_WRITE_REV_INDEX &&\n \t git_env_bool(\"GIT_TEST_MIDX_WRITE_REV\", 0))\n-\t\twrite_midx_reverse_index(&ctx, object_dir, midx_hash);\n+\t\twrite_midx_reverse_index(&ctx, source->path, midx_hash);\n \n \tif (flags & MIDX_WRITE_BITMAP) {\n \t\tstruct packing_data pdata;\n@@ -1414,7 +1406,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\tFREE_AND_NULL(ctx.entries);\n \t\tctx.entries_nr = 0;\n \n-\t\tif (write_midx_bitmap(&ctx, object_dir,\n+\t\tif (write_midx_bitmap(&ctx, source->path,\n \t\t\t\t midx_hash, &pdata, commits, commits_nr,\n \t\t\t\t flags) < 0) {\n \t\t\terror(_(\"could not write multi-pack bitmap\"));\n@@ -1448,7 +1440,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\t\treturn -1;\n \n \t\tget_split_midx_filename_ext(r->hash_algo, &final_midx_name,\n-\t\t\t\t\t object_dir, midx_hash, MIDX_EXT_MIDX);\n+\t\t\t\t\t source->path, midx_hash, MIDX_EXT_MIDX);\n \n \t\tif (rename_tempfile(&incr, final_midx_name.buf) < 0) {\n \t\t\terror_errno(_(\"unable to rename new multi-pack-index layer\"));\n@@ -1481,7 +1473,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (commit_lock_file(&lk) < 0)\n \t\tdie_errno(_(\"could not write multi-pack-index\"));\n \n-\tclear_midx_files(r, object_dir, keep_hashes,\n+\tclear_midx_files(source, keep_hashes,\n \t\t\t ctx.num_multi_pack_indexes_before + 1,\n \t\t\t ctx.incremental);\n \n@@ -1510,29 +1502,29 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \treturn result;\n }\n \n-int write_midx_file(struct repository *r, const char *object_dir,\n+int write_midx_file(struct odb_source *source,\n \t\t const char *preferred_pack_name,\n \t\t const char *refs_snapshot, unsigned flags)\n {\n-\treturn write_midx_internal(r, object_dir, NULL, NULL,\n+\treturn write_midx_internal(source, NULL, NULL,\n \t\t\t\t preferred_pack_name, refs_snapshot,\n \t\t\t\t flags);\n }\n \n-int write_midx_file_only(struct repository *r, const char *object_dir,\n+int write_midx_file_only(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags)\n {\n-\treturn write_midx_internal(r, object_dir, packs_to_include, NULL,\n+\treturn write_midx_internal(source, packs_to_include, NULL,\n \t\t\t\t preferred_pack_name, refs_snapshot, flags);\n }\n \n-int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags)\n+int expire_midx_packs(struct odb_source *source, unsigned flags)\n {\n \tuint32_t i, *count, result = 0;\n \tstruct string_list packs_to_drop = STRING_LIST_INIT_DUP;\n-\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \tstruct progress *progress = NULL;\n \n \tif (!m)\n@@ -1545,7 +1537,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \n \tif (flags & MIDX_PROGRESS)\n \t\tprogress = start_delayed_progress(\n-\t\t\t\t\t r,\n+\t\t\t\t\t source->odb->repo,\n \t\t\t\t\t _(\"Counting referenced objects\"),\n \t\t\t\t\t m->num_objects);\n \tfor (i = 0; i < m->num_objects; i++) {\n@@ -1557,7 +1549,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \n \tif (flags & MIDX_PROGRESS)\n \t\tprogress = start_delayed_progress(\n-\t\t\t\t\t r,\n+\t\t\t\t\t source->odb->repo,\n \t\t\t\t\t _(\"Finding and deleting unreferenced packfiles\"),\n \t\t\t\t\t m->num_packs);\n \tfor (i = 0; i < m->num_packs; i++) {\n@@ -1585,7 +1577,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \tfree(count);\n \n \tif (packs_to_drop.nr)\n-\t\tresult = write_midx_internal(r, object_dir, NULL,\n+\t\tresult = write_midx_internal(source, NULL,\n \t\t\t\t\t &packs_to_drop, NULL, NULL, flags);\n \n \tstring_list_clear(&packs_to_drop, 0);\n@@ -1710,14 +1702,15 @@ static void fill_included_packs_batch(struct repository *r,\n \tfree(pack_info);\n }\n \n-int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags)\n+int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tint result = 0;\n \tuint32_t i, packs_to_repack = 0;\n \tunsigned char *include_pack;\n \tstruct child_process cmd = CHILD_PROCESS_INIT;\n \tFILE *cmd_in;\n-\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \n \t/*\n \t * When updating the default for these configuration\n@@ -1751,7 +1744,7 @@ int midx_repack(struct repository *r, const char *object_dir, size_t batch_size,\n \n \tstrvec_push(&cmd.args, \"pack-objects\");\n \n-\tstrvec_pushf(&cmd.args, \"%s/pack/pack\", object_dir);\n+\tstrvec_pushf(&cmd.args, \"%s/pack/pack\", source->path);\n \n \tif (delta_base_offset)\n \t\tstrvec_push(&cmd.args, \"--delta-base-offset\");\n@@ -1792,7 +1785,7 @@ int midx_repack(struct repository *r, const char *object_dir, size_t batch_size,\n \t\tgoto cleanup;\n \t}\n \n-\tresult = write_midx_internal(r, object_dir, NULL, NULL, NULL, NULL,\n+\tresult = write_midx_internal(source, NULL, NULL, NULL, NULL,\n \t\t\t\t flags);\n \n cleanup:\ndiff --git a/midx.h b/midx.h\nindex 970d043989..d162001fbb 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -126,17 +126,17 @@ int prepare_multi_pack_index_one(struct odb_source *source);\n * Variant of write_midx_file which writes a MIDX containing only the packs\n * specified in packs_to_include.\n */\n-int write_midx_file(struct repository *r, const char *object_dir,\n+int write_midx_file(struct odb_source *source,\n \t\t const char *preferred_pack_name, const char *refs_snapshot,\n \t\t unsigned flags);\n-int write_midx_file_only(struct repository *r, const char *object_dir,\n+int write_midx_file_only(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags);\n void clear_midx_file(struct repository *r);\n int verify_midx_file(struct odb_source *source, unsigned flags);\n-int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags);\n-int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags);\n+int expire_midx_packs(struct odb_source *source, unsigned flags);\n+int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags);\n \n void close_midx(struct multi_pack_index *m);\n \n\n-- \n2.50.1.619.g074bbf1d35.dirty\n\n"},{"id":"522960","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-7-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH 7/8] midx: stop duplicating info redundant with its owning source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:46Z","receivedAt":"2025-07-29T14:13:06Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Multi-pack indices store some information that is redundant with their\nowning source:\n\n - The locality bit that tracks whether the source is the primary\n object source or an alternate.\n\n - The object directory path the multi-pack index is located in.\n\n - The pointer to the owning parent directory.\n\nAll of this information is already contained in `struct odb_source`. So\nnow that we always have that struct available when loading a multi-pack\nindex we have it readily accessible.\n\nDrop the redundant information and instead store a pointer to the object\nsource.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/repack.c | 5 +++--\n midx-write.c | 9 +++++----\n midx.c | 21 +++++++++++----------\n midx.h | 7 ++-----\n pack-bitmap.c | 13 +++++++------\n pack-revindex.c | 14 +++++++-------\n t/helper/test-read-midx.c | 2 +-\n 7 files changed, 36 insertions(+), 35 deletions(-)\n\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 94dec26f18..5af3e27357 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -223,9 +223,10 @@ static void mark_packs_for_deletion(struct existing_packs *existing,\n static void remove_redundant_pack(const char *dir_name, const char *base_name)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct multi_pack_index *m = get_multi_pack_index(the_repository->objects->sources);\n+\tstruct odb_source *source = the_repository->objects->sources;\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \tstrbuf_addf(&buf, \"%s.pack\", base_name);\n-\tif (m && m->local && midx_contains_pack(m, buf.buf))\n+\tif (m && source->local && midx_contains_pack(m, buf.buf))\n \t\tclear_midx_file(the_repository);\n \tstrbuf_insertf(&buf, 0, \"%s/\", dir_name);\n \tunlink_pack_path(buf.buf, 1);\ndiff --git a/midx-write.c b/midx-write.c\nindex bf7c01d4b1..84f76856d6 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -981,10 +981,11 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \tfor (i = 0; i < ARRAY_SIZE(midx_exts); i++) {\n \t\tconst unsigned char *hash = get_midx_checksum(m);\n \n-\t\tget_midx_filename_ext(m->repo->hash_algo, &from, m->object_dir,\n+\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &from,\n+\t\t\t\t m->source->path,\n \t\t\t\t hash, midx_exts[i].non_split);\n-\t\tget_split_midx_filename_ext(m->repo->hash_algo, &to,\n-\t\t\t\t\t m->object_dir, hash,\n+\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &to,\n+\t\t\t\t\t m->source->path, hash,\n \t\t\t\t\t midx_exts[i].split);\n \n \t\tif (link(from.buf, to.buf) < 0 && errno != ENOENT) {\n@@ -1109,7 +1110,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\tif (flags & MIDX_WRITE_BITMAP && load_midx_revindex(m)) {\n \t\t\t\terror(_(\"could not load reverse index for MIDX %s\"),\n \t\t\t\t hash_to_hex_algop(get_midx_checksum(m),\n-\t\t\t\t\t\t\tm->repo->hash_algo));\n+\t\t\t\t\t\t\tm->source->odb->repo->hash_algo));\n \t\t\t\tresult = 1;\n \t\t\t\tgoto cleanup;\n \t\t\t}\ndiff --git a/midx.c b/midx.c\nindex 831a7e9b5f..81bf3c4d5f 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -26,7 +26,7 @@ int cmp_idx_or_pack_name(const char *idx_or_pack_name,\n \n const unsigned char *get_midx_checksum(struct multi_pack_index *m)\n {\n-\treturn m->data + m->data_len - m->repo->hash_algo->rawsz;\n+\treturn m->data + m->data_len - m->source->odb->repo->hash_algo->rawsz;\n }\n \n void get_midx_filename(const struct git_hash_algo *hash_algo,\n@@ -128,11 +128,10 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou\n \tmidx_map = xmmap(NULL, midx_size, PROT_READ, MAP_PRIVATE, fd, 0);\n \tclose(fd);\n \n-\tFLEX_ALLOC_STR(m, object_dir, source->path);\n+\tCALLOC_ARRAY(m, 1);\n \tm->data = midx_map;\n \tm->data_len = midx_size;\n-\tm->local = source->local;\n-\tm->repo = r;\n+\tm->source = source;\n \n \tm->signature = get_be32(m->data);\n \tif (m->signature != MIDX_SIGNATURE)\n@@ -446,7 +445,7 @@ static uint32_t midx_for_pack(struct multi_pack_index **_m,\n int prepare_midx_pack(struct multi_pack_index *m,\n \t\t uint32_t pack_int_id)\n {\n-\tstruct repository *r = m->repo;\n+\tstruct repository *r = m->source->odb->repo;\n \tstruct strbuf pack_name = STRBUF_INIT;\n \tstruct strbuf key = STRBUF_INIT;\n \tstruct packed_git *p;\n@@ -458,7 +457,7 @@ int prepare_midx_pack(struct multi_pack_index *m,\n \tif (m->packs[pack_int_id])\n \t\treturn 0;\n \n-\tstrbuf_addf(&pack_name, \"%s/pack/%s\", m->object_dir,\n+\tstrbuf_addf(&pack_name, \"%s/pack/%s\", m->source->path,\n \t\t m->pack_names[pack_int_id]);\n \n \t/* pack_map holds the \".pack\" name, but we have the .idx */\n@@ -469,7 +468,8 @@ int prepare_midx_pack(struct multi_pack_index *m,\n \t\t\t\t\tstrhash(key.buf), key.buf,\n \t\t\t\t\tstruct packed_git, packmap_ent);\n \tif (!p) {\n-\t\tp = add_packed_git(r, pack_name.buf, pack_name.len, m->local);\n+\t\tp = add_packed_git(r, pack_name.buf, pack_name.len,\n+\t\t\t\t m->source->local);\n \t\tif (p) {\n \t\t\tinstall_packed_git(r, p);\n \t\t\tlist_add_tail(&p->mru, &r->objects->packed_git_mru);\n@@ -528,7 +528,8 @@ int bsearch_one_midx(const struct object_id *oid, struct multi_pack_index *m,\n \t\t uint32_t *result)\n {\n \tint ret = bsearch_hash(oid->hash, m->chunk_oid_fanout,\n-\t\t\t m->chunk_oid_lookup, m->repo->hash_algo->rawsz,\n+\t\t\t m->chunk_oid_lookup,\n+\t\t\t m->source->odb->repo->hash_algo->rawsz,\n \t\t\t result);\n \tif (result)\n \t\t*result += m->num_objects_in_base;\n@@ -559,7 +560,7 @@ struct object_id *nth_midxed_object_oid(struct object_id *oid,\n \tn = midx_for_object(&m, n);\n \n \toidread(oid, m->chunk_oid_lookup + st_mult(m->hash_len, n),\n-\t\tm->repo->hash_algo);\n+\t\tm->source->odb->repo->hash_algo);\n \treturn oid;\n }\n \n@@ -734,7 +735,7 @@ int prepare_multi_pack_index_one(struct odb_source *source)\n \n int midx_checksum_valid(struct multi_pack_index *m)\n {\n-\treturn hashfile_checksum_valid(m->repo->hash_algo,\n+\treturn hashfile_checksum_valid(m->source->odb->repo->hash_algo,\n \t\t\t\t m->data, m->data_len);\n }\n \ndiff --git a/midx.h b/midx.h\nindex d162001fbb..71dbdec66e 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -35,6 +35,8 @@ struct odb_source;\n \t\"GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL\"\n \n struct multi_pack_index {\n+\tstruct odb_source *source;\n+\n \tconst unsigned char *data;\n \tsize_t data_len;\n \n@@ -50,7 +52,6 @@ struct multi_pack_index {\n \tuint32_t num_objects;\n \tint preferred_pack_idx;\n \n-\tint local;\n \tint has_chain;\n \n \tconst unsigned char *chunk_pack_names;\n@@ -71,10 +72,6 @@ struct multi_pack_index {\n \n \tconst char **pack_names;\n \tstruct packed_git **packs;\n-\n-\tstruct repository *repo;\n-\n-\tchar object_dir[FLEX_ARRAY];\n };\n \n #define MIDX_PROGRESS (1 << 0)\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex fb0b11ca07..01e14c34bd 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -216,7 +216,7 @@ static uint32_t bitmap_num_objects(struct bitmap_index *index)\n static struct repository *bitmap_repo(struct bitmap_index *bitmap_git)\n {\n \tif (bitmap_is_midx(bitmap_git))\n-\t\treturn bitmap_git->midx->repo;\n+\t\treturn bitmap_git->midx->source->odb->repo;\n \treturn bitmap_git->pack->repo;\n }\n \n@@ -418,13 +418,13 @@ char *midx_bitmap_filename(struct multi_pack_index *midx)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n \tif (midx->has_chain)\n-\t\tget_split_midx_filename_ext(midx->repo->hash_algo, &buf,\n-\t\t\t\t\t midx->object_dir,\n+\t\tget_split_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t\t midx->source->path,\n \t\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t\t MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(midx->repo->hash_algo, &buf,\n-\t\t\t\t midx->object_dir, get_midx_checksum(midx),\n+\t\tget_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t midx->source->path, get_midx_checksum(midx),\n \t\t\t\t MIDX_EXT_BITMAP);\n \n \treturn strbuf_detach(&buf, NULL);\n@@ -463,7 +463,8 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tget_midx_filename(midx->repo->hash_algo, &buf, midx->object_dir);\n+\t\tget_midx_filename(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t midx->source->path);\n \t\ttrace2_data_string(\"bitmap\", bitmap_repo(bitmap_git),\n \t\t\t\t \"ignoring extra midx bitmap file\", buf.buf);\n \t\tclose(fd);\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 0cc422a1e6..b206518dcb 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -379,25 +379,25 @@ int load_midx_revindex(struct multi_pack_index *m)\n \t\t * not want to accidentally call munmap() in the middle of the\n \t\t * MIDX.\n \t\t */\n-\t\ttrace2_data_string(\"load_midx_revindex\", m->repo,\n+\t\ttrace2_data_string(\"load_midx_revindex\", m->source->odb->repo,\n \t\t\t\t \"source\", \"midx\");\n \t\tm->revindex_data = (const uint32_t *)m->chunk_revindex;\n \t\treturn 0;\n \t}\n \n-\ttrace2_data_string(\"load_midx_revindex\", m->repo,\n+\ttrace2_data_string(\"load_midx_revindex\", m->source->odb->repo,\n \t\t\t \"source\", \"rev\");\n \n \tif (m->has_chain)\n-\t\tget_split_midx_filename_ext(m->repo->hash_algo, &revindex_name,\n-\t\t\t\t\t m->object_dir, get_midx_checksum(m),\n+\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n+\t\t\t\t\t m->source->path, get_midx_checksum(m),\n \t\t\t\t\t MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(m->repo->hash_algo, &revindex_name,\n-\t\t\t\t m->object_dir, get_midx_checksum(m),\n+\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n+\t\t\t\t m->source->path, get_midx_checksum(m),\n \t\t\t\t MIDX_EXT_REV);\n \n-\tret = load_revindex_from_disk(m->repo->hash_algo,\n+\tret = load_revindex_from_disk(m->source->odb->repo->hash_algo,\n \t\t\t\t revindex_name.buf,\n \t\t\t\t m->num_objects,\n \t\t\t\t &m->revindex_map,\ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex bcb8ea7671..6de5d1665a 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -66,7 +66,7 @@ static int read_midx_file(const char *object_dir, const char *checksum,\n \tfor (i = 0; i < m->num_packs; i++)\n \t\tprintf(\"%s\\n\", m->pack_names[i]);\n \n-\tprintf(\"object-dir: %s\\n\", m->object_dir);\n+\tprintf(\"object-dir: %s\\n\", m->source->path);\n \n \tif (show_objects) {\n \t\tstruct object_id oid;\n\n-- \n2.50.1.619.g074bbf1d35.dirty\n\n"},{"id":"522961","messageId":"20250729-b4-pks-midx-deduplicate-source-info-v1-8-748db2eda3b5@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH 8/8] midx: compute paths via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-29T14:12:47Z","receivedAt":"2025-07-29T14:13:08Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"With the preceding commits we started to always have the object database\nsource available when we load, write or access multi-pack indices. With\nthis in place we can change how MIDX paths are computed so that we don't\nhave to pass in the combination of a hash algorithm and object directory\nanymore, but only the object database source.\n\nRefactor the code accordingly.\n\nSigned-off-by: Patrick Steinhardt \n---\n midx-write.c | 52 ++++++++++++++++++++++++----------------------------\n midx.c | 54 +++++++++++++++++++++++++-----------------------------\n midx.h | 13 +++++--------\n pack-bitmap.c | 10 ++++------\n pack-revindex.c | 8 ++++----\n 5 files changed, 62 insertions(+), 75 deletions(-)\n\ndiff --git a/midx-write.c b/midx-write.c\nindex 84f76856d6..1dcdf3dc0f 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -26,9 +26,9 @@\n #define MIDX_CHUNK_LARGE_OFFSET_WIDTH (sizeof(uint64_t))\n \n extern int midx_checksum_valid(struct multi_pack_index *m);\n-extern void clear_midx_files_ext(const char *object_dir, const char *ext,\n+extern void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t const char *keep_hash);\n-extern void clear_incremental_midx_files_ext(const char *object_dir,\n+extern void clear_incremental_midx_files_ext(struct odb_source *source,\n \t\t\t\t\t const char *ext,\n \t\t\t\t\t const char **keep_hashes,\n \t\t\t\t\t uint32_t hashes_nr);\n@@ -112,6 +112,7 @@ struct write_midx_context {\n \tstruct string_list *to_include;\n \n \tstruct repository *repo;\n+\tstruct odb_source *source;\n };\n \n static int should_include_pack(const struct write_midx_context *ctx,\n@@ -648,7 +649,6 @@ static uint32_t *midx_pack_order(struct write_midx_context *ctx)\n }\n \n static void write_midx_reverse_index(struct write_midx_context *ctx,\n-\t\t\t\t const char *object_dir,\n \t\t\t\t unsigned char *midx_hash)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -657,11 +657,10 @@ static void write_midx_reverse_index(struct write_midx_context *ctx,\n \ttrace2_region_enter(\"midx\", \"write_midx_reverse_index\", ctx->repo);\n \n \tif (ctx->incremental)\n-\t\tget_split_midx_filename_ext(ctx->repo->hash_algo, &buf,\n-\t\t\t\t\t object_dir, midx_hash,\n-\t\t\t\t\t MIDX_EXT_REV);\n+\t\tget_split_midx_filename_ext(ctx->source, &buf,\n+\t\t\t\t\t midx_hash, MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(ctx->repo->hash_algo, &buf, object_dir,\n+\t\tget_midx_filename_ext(ctx->source, &buf,\n \t\t\t\t midx_hash, MIDX_EXT_REV);\n \n \ttmp_file = write_rev_file_order(ctx->repo, NULL, ctx->pack_order,\n@@ -836,7 +835,6 @@ static struct commit **find_commits_for_midx_bitmap(uint32_t *indexed_commits_nr\n }\n \n static int write_midx_bitmap(struct write_midx_context *ctx,\n-\t\t\t const char *object_dir,\n \t\t\t const unsigned char *midx_hash,\n \t\t\t struct packing_data *pdata,\n \t\t\t struct commit **commits,\n@@ -852,12 +850,11 @@ static int write_midx_bitmap(struct write_midx_context *ctx,\n \ttrace2_region_enter(\"midx\", \"write_midx_bitmap\", ctx->repo);\n \n \tif (ctx->incremental)\n-\t\tget_split_midx_filename_ext(ctx->repo->hash_algo, &bitmap_name,\n-\t\t\t\t\t object_dir, midx_hash,\n-\t\t\t\t\t MIDX_EXT_BITMAP);\n+\t\tget_split_midx_filename_ext(ctx->source, &bitmap_name,\n+\t\t\t\t\t midx_hash, MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(ctx->repo->hash_algo, &bitmap_name,\n-\t\t\t\t object_dir, midx_hash, MIDX_EXT_BITMAP);\n+\t\tget_midx_filename_ext(ctx->source, &bitmap_name,\n+\t\t\t\t midx_hash, MIDX_EXT_BITMAP);\n \n \tif (flags & MIDX_WRITE_BITMAP_HASH_CACHE)\n \t\toptions |= BITMAP_OPT_HASH_CACHE;\n@@ -981,11 +978,9 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \tfor (i = 0; i < ARRAY_SIZE(midx_exts); i++) {\n \t\tconst unsigned char *hash = get_midx_checksum(m);\n \n-\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &from,\n-\t\t\t\t m->source->path,\n+\t\tget_midx_filename_ext(m->source, &from,\n \t\t\t\t hash, midx_exts[i].non_split);\n-\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &to,\n-\t\t\t\t\t m->source->path, hash,\n+\t\tget_split_midx_filename_ext(m->source, &to, hash,\n \t\t\t\t\t midx_exts[i].split);\n \n \t\tif (link(from.buf, to.buf) < 0 && errno != ENOENT) {\n@@ -1023,16 +1018,16 @@ static void clear_midx_files(struct odb_source *source,\n \tuint32_t i, j;\n \n \tfor (i = 0; i < ARRAY_SIZE(exts); i++) {\n-\t\tclear_incremental_midx_files_ext(source->path, exts[i],\n+\t\tclear_incremental_midx_files_ext(source, exts[i],\n \t\t\t\t\t\t hashes, hashes_nr);\n \t\tfor (j = 0; j < hashes_nr; j++)\n-\t\t\tclear_midx_files_ext(source->path, exts[i], hashes[j]);\n+\t\t\tclear_midx_files_ext(source, exts[i], hashes[j]);\n \t}\n \n \tif (incremental)\n-\t\tget_midx_filename(source->odb->repo->hash_algo, &buf, source->path);\n+\t\tget_midx_filename(source, &buf);\n \telse\n-\t\tget_midx_chain_filename(&buf, source->path);\n+\t\tget_midx_chain_filename(source, &buf);\n \n \tif (unlink(buf.buf) && errno != ENOENT)\n \t\tdie_errno(_(\"failed to clear multi-pack-index at %s\"), buf.buf);\n@@ -1065,6 +1060,7 @@ static int write_midx_internal(struct odb_source *source,\n \ttrace2_region_enter(\"midx\", \"write_midx_internal\", r);\n \n \tctx.repo = r;\n+\tctx.source = source;\n \n \tctx.incremental = !!(flags & MIDX_WRITE_INCREMENTAL);\n \n@@ -1073,7 +1069,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\t \"%s/pack/multi-pack-index.d/tmp_midx_XXXXXX\",\n \t\t\t source->path);\n \telse\n-\t\tget_midx_filename(r->hash_algo, &midx_name, source->path);\n+\t\tget_midx_filename(source, &midx_name);\n \tif (safe_create_leading_directories(r, midx_name.buf))\n \t\tdie_errno(_(\"unable to create leading directories of %s\"),\n \t\t\t midx_name.buf);\n@@ -1153,7 +1149,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\t * corresponding bitmap (or one wasn't requested).\n \t\t\t */\n \t\t\tif (!want_bitmap)\n-\t\t\t\tclear_midx_files_ext(source->path, \"bitmap\", NULL);\n+\t\t\t\tclear_midx_files_ext(source, \"bitmap\", NULL);\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -1321,7 +1317,7 @@ static int write_midx_internal(struct odb_source *source,\n \tif (ctx.incremental) {\n \t\tstruct strbuf lock_name = STRBUF_INIT;\n \n-\t\tget_midx_chain_filename(&lock_name, source->path);\n+\t\tget_midx_chain_filename(source, &lock_name);\n \t\thold_lock_file_for_update(&lk, lock_name.buf, LOCK_DIE_ON_ERROR);\n \t\tstrbuf_release(&lock_name);\n \n@@ -1384,7 +1380,7 @@ static int write_midx_internal(struct odb_source *source,\n \n \tif (flags & MIDX_WRITE_REV_INDEX &&\n \t git_env_bool(\"GIT_TEST_MIDX_WRITE_REV\", 0))\n-\t\twrite_midx_reverse_index(&ctx, source->path, midx_hash);\n+\t\twrite_midx_reverse_index(&ctx, midx_hash);\n \n \tif (flags & MIDX_WRITE_BITMAP) {\n \t\tstruct packing_data pdata;\n@@ -1407,7 +1403,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\tFREE_AND_NULL(ctx.entries);\n \t\tctx.entries_nr = 0;\n \n-\t\tif (write_midx_bitmap(&ctx, source->path,\n+\t\tif (write_midx_bitmap(&ctx,\n \t\t\t\t midx_hash, &pdata, commits, commits_nr,\n \t\t\t\t flags) < 0) {\n \t\t\terror(_(\"could not write multi-pack bitmap\"));\n@@ -1440,8 +1436,8 @@ static int write_midx_internal(struct odb_source *source,\n \t\tif (link_midx_to_chain(ctx.base_midx) < 0)\n \t\t\treturn -1;\n \n-\t\tget_split_midx_filename_ext(r->hash_algo, &final_midx_name,\n-\t\t\t\t\t source->path, midx_hash, MIDX_EXT_MIDX);\n+\t\tget_split_midx_filename_ext(source, &final_midx_name,\n+\t\t\t\t\t midx_hash, MIDX_EXT_MIDX);\n \n \t\tif (rename_tempfile(&incr, final_midx_name.buf) < 0) {\n \t\t\terror_errno(_(\"unable to rename new multi-pack-index layer\"));\ndiff --git a/midx.c b/midx.c\nindex 81bf3c4d5f..7726c13d7e 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -16,9 +16,9 @@\n #define MIDX_PACK_ERROR ((void *)(intptr_t)-1)\n \n int midx_checksum_valid(struct multi_pack_index *m);\n-void clear_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t const char *keep_hash);\n-void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t char **keep_hashes,\n \t\t\t\t uint32_t hashes_nr);\n int cmp_idx_or_pack_name(const char *idx_or_pack_name,\n@@ -29,19 +29,17 @@ const unsigned char *get_midx_checksum(struct multi_pack_index *m)\n \treturn m->data + m->data_len - m->source->odb->repo->hash_algo->rawsz;\n }\n \n-void get_midx_filename(const struct git_hash_algo *hash_algo,\n-\t\t struct strbuf *out, const char *object_dir)\n+void get_midx_filename(struct odb_source *source, struct strbuf *out)\n {\n-\tget_midx_filename_ext(hash_algo, out, object_dir, NULL, NULL);\n+\tget_midx_filename_ext(source, out, NULL, NULL);\n }\n \n-void get_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t struct strbuf *out, const char *object_dir,\n+void get_midx_filename_ext(struct odb_source *source, struct strbuf *out,\n \t\t\t const unsigned char *hash, const char *ext)\n {\n-\tstrbuf_addf(out, \"%s/pack/multi-pack-index\", object_dir);\n+\tstrbuf_addf(out, \"%s/pack/multi-pack-index\", source->path);\n \tif (ext)\n-\t\tstrbuf_addf(out, \"-%s.%s\", hash_to_hex_algop(hash, hash_algo), ext);\n+\t\tstrbuf_addf(out, \"-%s.%s\", hash_to_hex_algop(hash, source->odb->repo->hash_algo), ext);\n }\n \n static int midx_read_oid_fanout(const unsigned char *chunk_start,\n@@ -222,24 +220,23 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou\n \treturn NULL;\n }\n \n-void get_midx_chain_dirname(struct strbuf *buf, const char *object_dir)\n+void get_midx_chain_dirname(struct odb_source *source, struct strbuf *buf)\n {\n-\tstrbuf_addf(buf, \"%s/pack/multi-pack-index.d\", object_dir);\n+\tstrbuf_addf(buf, \"%s/pack/multi-pack-index.d\", source->path);\n }\n \n-void get_midx_chain_filename(struct strbuf *buf, const char *object_dir)\n+void get_midx_chain_filename(struct odb_source *source, struct strbuf *buf)\n {\n-\tget_midx_chain_dirname(buf, object_dir);\n+\tget_midx_chain_dirname(source, buf);\n \tstrbuf_addstr(buf, \"/multi-pack-index-chain\");\n }\n \n-void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t\t struct strbuf *buf, const char *object_dir,\n+void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,\n \t\t\t\t const unsigned char *hash, const char *ext)\n {\n-\tget_midx_chain_dirname(buf, object_dir);\n+\tget_midx_chain_dirname(source, buf);\n \tstrbuf_addf(buf, \"/multi-pack-index-%s.%s\",\n-\t\t hash_to_hex_algop(hash, hash_algo), ext);\n+\t\t hash_to_hex_algop(hash, source->odb->repo->hash_algo), ext);\n }\n \n static int open_multi_pack_index_chain(const struct git_hash_algo *hash_algo,\n@@ -326,7 +323,7 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source *source,\n \t\tvalid = 0;\n \n \t\tstrbuf_reset(&buf);\n-\t\tget_split_midx_filename_ext(hash_algo, &buf, source->path,\n+\t\tget_split_midx_filename_ext(source, &buf,\n \t\t\t\t\t layer.hash, MIDX_EXT_MIDX);\n \t\tm = load_multi_pack_index_one(source, buf.buf);\n \n@@ -358,7 +355,7 @@ static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *s\n \tint fd;\n \tstruct multi_pack_index *m = NULL;\n \n-\tget_midx_chain_filename(&chain_file, source->path);\n+\tget_midx_chain_filename(source, &chain_file);\n \tif (open_multi_pack_index_chain(source->odb->repo->hash_algo, chain_file.buf, &fd, &st)) {\n \t\tint incomplete;\n \t\t/* ownership of fd is taken over by load function */\n@@ -374,8 +371,7 @@ struct multi_pack_index *load_multi_pack_index(struct odb_source *source)\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tstruct multi_pack_index *m;\n \n-\tget_midx_filename(source->odb->repo->hash_algo, &midx_name,\n-\t\t\t source->path);\n+\tget_midx_filename(source, &midx_name);\n \n \tm = load_multi_pack_index_one(source, midx_name.buf);\n \tif (!m)\n@@ -762,7 +758,7 @@ static void clear_midx_file_ext(const char *full_path, size_t full_path_len UNUS\n \t\tdie_errno(_(\"failed to remove %s\"), full_path);\n }\n \n-void clear_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t const char *keep_hash)\n {\n \tstruct clear_midx_data data;\n@@ -776,7 +772,7 @@ void clear_midx_files_ext(const char *object_dir, const char *ext,\n \t}\n \tdata.ext = ext;\n \n-\tfor_each_file_in_pack_dir(object_dir,\n+\tfor_each_file_in_pack_dir(source->path,\n \t\t\t\t clear_midx_file_ext,\n \t\t\t\t &data);\n \n@@ -785,7 +781,7 @@ void clear_midx_files_ext(const char *object_dir, const char *ext,\n \tfree(data.keep);\n }\n \n-void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t char **keep_hashes,\n \t\t\t\t uint32_t hashes_nr)\n {\n@@ -801,7 +797,7 @@ void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n \tdata.keep_nr = hashes_nr;\n \tdata.ext = ext;\n \n-\tfor_each_file_in_pack_subdir(object_dir, \"multi-pack-index.d\",\n+\tfor_each_file_in_pack_subdir(source->path, \"multi-pack-index.d\",\n \t\t\t\t clear_midx_file_ext, &data);\n \n \tfor (i = 0; i < hashes_nr; i++)\n@@ -813,7 +809,7 @@ void clear_midx_file(struct repository *r)\n {\n \tstruct strbuf midx = STRBUF_INIT;\n \n-\tget_midx_filename(r->hash_algo, &midx, r->objects->sources->path);\n+\tget_midx_filename(r->objects->sources, &midx);\n \n \tif (r->objects) {\n \t\tstruct odb_source *source;\n@@ -828,8 +824,8 @@ void clear_midx_file(struct repository *r)\n \tif (remove_path(midx.buf))\n \t\tdie(_(\"failed to clear multi-pack-index at %s\"), midx.buf);\n \n-\tclear_midx_files_ext(r->objects->sources->path, MIDX_EXT_BITMAP, NULL);\n-\tclear_midx_files_ext(r->objects->sources->path, MIDX_EXT_REV, NULL);\n+\tclear_midx_files_ext(r->objects->sources, MIDX_EXT_BITMAP, NULL);\n+\tclear_midx_files_ext(r->objects->sources, MIDX_EXT_REV, NULL);\n \n \tstrbuf_release(&midx);\n }\n@@ -888,7 +884,7 @@ int verify_midx_file(struct odb_source *source, unsigned flags)\n \t\tstruct stat sb;\n \t\tstruct strbuf filename = STRBUF_INIT;\n \n-\t\tget_midx_filename(r->hash_algo, &filename, source->path);\n+\t\tget_midx_filename(source, &filename);\n \n \t\tif (!stat(filename.buf, &sb)) {\n \t\t\terror(_(\"multi-pack-index file exists, but failed to parse\"));\ndiff --git a/midx.h b/midx.h\nindex 71dbdec66e..e241d2d690 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -86,15 +86,12 @@ struct multi_pack_index {\n #define MIDX_EXT_MIDX \"midx\"\n \n const unsigned char *get_midx_checksum(struct multi_pack_index *m);\n-void get_midx_filename(const struct git_hash_algo *hash_algo,\n-\t\t struct strbuf *out, const char *object_dir);\n-void get_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t struct strbuf *out, const char *object_dir,\n+void get_midx_filename(struct odb_source *source, struct strbuf *out);\n+void get_midx_filename_ext(struct odb_source *source, struct strbuf *out,\n \t\t\t const unsigned char *hash, const char *ext);\n-void get_midx_chain_dirname(struct strbuf *buf, const char *object_dir);\n-void get_midx_chain_filename(struct strbuf *buf, const char *object_dir);\n-void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t\t struct strbuf *buf, const char *object_dir,\n+void get_midx_chain_dirname(struct odb_source *source, struct strbuf *out);\n+void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);\n+void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,\n \t\t\t\t const unsigned char *hash, const char *ext);\n \n struct multi_pack_index *load_multi_pack_index(struct odb_source *source);\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 01e14c34bd..058bdb5d7d 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -418,13 +418,12 @@ char *midx_bitmap_filename(struct multi_pack_index *midx)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n \tif (midx->has_chain)\n-\t\tget_split_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t\t midx->source->path,\n+\t\tget_split_midx_filename_ext(midx->source, &buf,\n \t\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t\t MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t midx->source->path, get_midx_checksum(midx),\n+\t\tget_midx_filename_ext(midx->source, &buf,\n+\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t MIDX_EXT_BITMAP);\n \n \treturn strbuf_detach(&buf, NULL);\n@@ -463,8 +462,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tget_midx_filename(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t midx->source->path);\n+\t\tget_midx_filename(midx->source, &buf);\n \t\ttrace2_data_string(\"bitmap\", bitmap_repo(bitmap_git),\n \t\t\t\t \"ignoring extra midx bitmap file\", buf.buf);\n \t\tclose(fd);\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex b206518dcb..d0791cc493 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -389,12 +389,12 @@ int load_midx_revindex(struct multi_pack_index *m)\n \t\t\t \"source\", \"rev\");\n \n \tif (m->has_chain)\n-\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n-\t\t\t\t\t m->source->path, get_midx_checksum(m),\n+\t\tget_split_midx_filename_ext(m->source, &revindex_name,\n+\t\t\t\t\t get_midx_checksum(m),\n \t\t\t\t\t MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n-\t\t\t\t m->source->path, get_midx_checksum(m),\n+\t\tget_midx_filename_ext(m->source, &revindex_name,\n+\t\t\t\t get_midx_checksum(m),\n \t\t\t\t MIDX_EXT_REV);\n \n \tret = load_revindex_from_disk(m->source->odb->repo->hash_algo,\n\n-- \n2.50.1.619.g074bbf1d35.dirty\n\n"},{"id":"522979","messageId":"xmqqwm7qonhq.fsf@gitster.g","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"Re: [PATCH 0/8] midx: stop deduplicating info redundant with their sources","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-07-29T18:33:53Z","receivedAt":"2025-07-29T18:33:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Hi,\n>\n> this patch series is a follow up to the recent patch series that started\n> to move the MIDX into its object source. It refactors the MIDX subsystem\n> so that we always read and write them via an ODB source. This ultimately\n> allows us to store a pointer to the owning ODB source in the MIDX and\n> thus dedup some redundant information.\n\n\"Stop deduplicating info\" in the series title is probably a typo of\n\"stop duplicating\" (which is the title of the 7th step)?\n"},{"id":"522991","messageId":"aImrxAk3PGdUSfYe@pks.im","threadId":"63871","inReplyTo":"xmqqwm7qonhq.fsf@gitster.g","subject":"Re: [PATCH 0/8] midx: stop deduplicating info redundant with their sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-07-30T05:21:08Z","receivedAt":"2025-07-30T05:21:15Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Tue, Jul 29, 2025 at 11:33:53AM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n> > this patch series is a follow up to the recent patch series that started\n> > to move the MIDX into its object source. It refactors the MIDX subsystem\n> > so that we always read and write them via an ODB source. This ultimately\n> > allows us to store a pointer to the owning ODB source in the MIDX and\n> > thus dedup some redundant information.\n> \n> \"Stop deduplicating info\" in the series title is probably a typo of\n> \"stop duplicating\" (which is the title of the 7th step)?\n\nOh, indeed. Will fix in the next iteration, thanks!\n\nPatrick\n"},{"id":"523664","messageId":"87fre4tndx.fsf@iotcl.com","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-1-748db2eda3b5@pks.im","subject":"Re: [PATCH 1/8] odb: store locality in object database sources","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2025-08-06T16:39:54Z","receivedAt":"2025-08-06T16:40:10Z","isPatch":true,"sender":{"key":"toon@iotcl.com","avatar":"https://avatars.githubusercontent.com/u/121621?v=4"},"body":"Patrick Steinhardt writes:\n\n> Object database sources are classified either as:\n>\n> - Local, which means that the source is the repository's primary\n> source. This is typically \".git/objects\".\n>\n> - Non-local, which is everything else. Most importantly this includes\n> alternates and quarantine directories.\n>\n> This locality is often computed ad-hoc by checking whether a given\n> object source is the first one. This works, but it is quite roundabout.\n>\n> Refactor the code so that we store locality when creating the sources in\n> the first place. This makes it both more accessible and robust.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> midx.c | 5 +++--\n> midx.h | 2 +-\n> odb.c | 1 +\n> odb.h | 8 ++++++++\n> packfile.c | 9 ++++-----\n> repository.c | 1 +\n> 6 files changed, 18 insertions(+), 8 deletions(-)\n>\n> diff --git a/midx.c b/midx.c\n> index 7d407682e6..b9ca0915a6 100644\n> --- a/midx.c\n> +++ b/midx.c\n> @@ -723,7 +723,7 @@ int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id)\n> \treturn 0;\n> }\n> \n> -int prepare_multi_pack_index_one(struct odb_source *source, int local)\n> +int prepare_multi_pack_index_one(struct odb_source *source)\n> {\n> \tstruct repository *r = source->odb->repo;\n> \n> @@ -734,7 +734,8 @@ int prepare_multi_pack_index_one(struct odb_source *source, int local)\n> \tif (source->midx)\n> \t\treturn 1;\n> \n> -\tsource->midx = load_multi_pack_index(r, source->path, local);\n> +\tsource->midx = load_multi_pack_index(r, source->path,\n> +\t\t\t\t\t source->local);\n> \n> \treturn !!source->midx;\n> }\n> diff --git a/midx.h b/midx.h\n> index 076382de8a..28c426a823 100644\n> --- a/midx.h\n> +++ b/midx.h\n> @@ -122,7 +122,7 @@ int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pa\n> int midx_contains_pack(struct multi_pack_index *m,\n> \t\t const char *idx_or_pack_name);\n> int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);\n> -int prepare_multi_pack_index_one(struct odb_source *source, int local);\n> +int prepare_multi_pack_index_one(struct odb_source *source);\n> \n> /*\n> * Variant of write_midx_file which writes a MIDX containing only the packs\n> diff --git a/odb.c b/odb.c\n> index 1f48a0448e..1761a50840 100644\n> --- a/odb.c\n> +++ b/odb.c\n> @@ -176,6 +176,7 @@ static int link_alt_odb_entry(struct object_database *odb,\n> \n> \tCALLOC_ARRAY(alternate, 1);\n> \talternate->odb = odb;\n> +\talternate->local = false;\n> \t/* pathbuf.buf is already in r->objects->source_by_path */\n> \talternate->path = strbuf_detach(&pathbuf, NULL);\n> \n> diff --git a/odb.h b/odb.h\n> index 09177bf430..d9f4dcf79f 100644\n> --- a/odb.h\n> +++ b/odb.h\n> @@ -63,6 +63,14 @@ struct odb_source {\n> \t */\n> \tstruct multi_pack_index *midx;\n> \n> +\t/*\n> +\t * Figure out whether this is the local alternate of the owning\n\nDo we still use \"alternate\" in this context? Shouldn't it be \"source\"?\n\n-- \nCheers,\nToon\n"},{"id":"523665","messageId":"87ectotnd4.fsf@iotcl.com","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-3-748db2eda3b5@pks.im","subject":"Re: [PATCH 3/8] odb: return newly created in-memory sources","fromName":"Toon Claes","fromEmail":"toon@iotcl.com","sentAt":"2025-08-06T16:40:23Z","receivedAt":"2025-08-06T16:40:35Z","isPatch":true,"sender":{"key":"toon@iotcl.com","avatar":"https://avatars.githubusercontent.com/u/121621?v=4"},"body":"Patrick Steinhardt writes:\n\n> Callers have no trivial way to obtain the newly created object database\n> source when adding it to the in-memory list of alternates. While not yet\n> needed anywhere, a subsequent commit will want to obtain that pointer.\n>\n> Refactor the function to return the source to make it easily accessible.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> odb.c | 37 +++++++++++++++++++++----------------\n> odb.h | 4 ++--\n> 2 files changed, 23 insertions(+), 18 deletions(-)\n>\n> diff --git a/odb.c b/odb.c\n> index 61104b7cb8..7793816f81 100644\n> --- a/odb.c\n> +++ b/odb.c\n> @@ -139,23 +139,22 @@ static void read_info_alternates(struct object_database *odb,\n> \t\t\t\t const char *relative_base,\n> \t\t\t\t int depth);\n> \n> -static int link_alt_odb_entry(struct object_database *odb,\n> -\t\t\t const struct strbuf *entry,\n> -\t\t\t const char *relative_base,\n> -\t\t\t int depth,\n> -\t\t\t const char *normalized_objdir)\n> +static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n> +\t\t\t\t\t const char *entry,\n> +\t\t\t\t\t const char *relative_base,\n> +\t\t\t\t\t int depth,\n> +\t\t\t\t\t const char *normalized_objdir)\n> {\n> -\tstruct odb_source *alternate;\n> +\tstruct odb_source *alternate = NULL;\n> \tstruct strbuf pathbuf = STRBUF_INIT;\n> \tstruct strbuf tmp = STRBUF_INIT;\n> \tkhiter_t pos;\n> -\tint ret = -1;\n> \n> -\tif (!is_absolute_path(entry->buf) && relative_base) {\n> +\tif (!is_absolute_path(entry) && relative_base) {\n> \t\tstrbuf_realpath(&pathbuf, relative_base, 1);\n> \t\tstrbuf_addch(&pathbuf, '/');\n> \t}\n> -\tstrbuf_addbuf(&pathbuf, entry);\n> +\tstrbuf_addstr(&pathbuf, entry);\n> \n> \tif (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {\n> \t\terror(_(\"unable to normalize alternate object path: %s\"),\n> @@ -189,11 +188,11 @@ static int link_alt_odb_entry(struct object_database *odb,\n> \n> \t/* recursively add alternates */\n> \tread_info_alternates(odb, alternate->path, depth + 1);\n> -\tret = 0;\n> +\n> error:\n> \tstrbuf_release(&tmp);\n> \tstrbuf_release(&pathbuf);\n> -\treturn ret;\n> +\treturn alternate;\n> }\n> \n> static const char *parse_alt_odb_entry(const char *string,\n> @@ -246,7 +245,7 @@ static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n> \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n> \t\tif (!entry.len)\n> \t\t\tcontinue;\n> -\t\tlink_alt_odb_entry(odb, &entry,\n> +\t\tlink_alt_odb_entry(odb, entry.buf,\n> \t\t\t\t relative_base, depth, objdirbuf.buf);\n> \t}\n> \tstrbuf_release(&entry);\n> @@ -316,17 +315,23 @@ void odb_add_to_alternates_file(struct object_database *odb,\n> \tfree(alts);\n> }\n> \n> -void odb_add_to_alternates_memory(struct object_database *odb,\n> -\t\t\t\t const char *reference)\n> +struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n> +\t\t\t\t\t\tconst char *reference)\n> {\n> +\tstruct odb_source *alternate;\n> +\tchar *objdir;\n> +\n> \t/*\n> \t * Make sure alternates are initialized, or else our entry may be\n> \t * overwritten when they are.\n> \t */\n> \todb_prepare_alternates(odb);\n> \n> -\tlink_alt_odb_entries(odb, reference,\n> -\t\t\t '\\n', NULL, 0);\n> +\tobjdir = real_pathdup(odb->sources->path, 1);\n> +\talternate = link_alt_odb_entry(odb, reference, NULL, 0, objdir);\n\nIf I understand correctly, instead of using real_pathdup() we could\ninstead call:\n\n alternate = link_alt_odb_entry(odb, reference, \"/\", 0, odb->sources->path);\n\nI did not test this, but it would avoid duplicating the path here. I'm\nnot sure though whether it's easier to read.\n\n> +\n> +\tfree(objdir);\n> +\treturn alternate;\n> }\n> \n> struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,\n> diff --git a/odb.h b/odb.h\n> index 387b117c87..ac7ee05188 100644\n> --- a/odb.h\n> +++ b/odb.h\n> @@ -265,8 +265,8 @@ void odb_add_to_alternates_file(struct object_database *odb,\n> * recursive alternates it points to), but do not modify the on-disk alternates\n> * file.\n> */\n> -void odb_add_to_alternates_memory(struct object_database *odb,\n> -\t\t\t\t const char *dir);\n> +struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n> +\t\t\t\t\t\tconst char *dir);\n> \n> /*\n> * Read an object from the database. Returns the object data and assigns object\n>\n> -- \n> 2.50.1.619.g074bbf1d35.dirty\n>\n>\n\n-- \nCheers,\nToon\n"},{"id":"523707","messageId":"aJRElp0_hWlWz-5a@pks.im","threadId":"63871","inReplyTo":"87fre4tndx.fsf@iotcl.com","subject":"Re: [PATCH 1/8] odb: store locality in object database sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T06:15:50Z","receivedAt":"2025-08-07T06:15:55Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Aug 06, 2025 at 06:39:54PM +0200, Toon Claes wrote:\n> Patrick Steinhardt writes:\n> > diff --git a/odb.h b/odb.h\n> > index 09177bf430..d9f4dcf79f 100644\n> > --- a/odb.h\n> > +++ b/odb.h\n> > @@ -63,6 +63,14 @@ struct odb_source {\n> > \t */\n> > \tstruct multi_pack_index *midx;\n> > \n> > +\t/*\n> > +\t * Figure out whether this is the local alternate of the owning\n> \n> Do we still use \"alternate\" in this context? Shouldn't it be \"source\"?\n\nGood catch, it should be source indeed. We only talk about alternates\nwhen we actually retrieve them from \".git/objects/info/alternates\".\n\nPatrick\n"},{"id":"523708","messageId":"aJREtxNAJ9tK30GT@pks.im","threadId":"63871","inReplyTo":"87ectotnd4.fsf@iotcl.com","subject":"Re: [PATCH 3/8] odb: return newly created in-memory sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T06:16:23Z","receivedAt":"2025-08-07T06:16:29Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Wed, Aug 06, 2025 at 06:40:23PM +0200, Toon Claes wrote:\n> Patrick Steinhardt writes:\n> > diff --git a/odb.c b/odb.c\n> > index 61104b7cb8..7793816f81 100644\n> > --- a/odb.c\n> > +++ b/odb.c\n> > @@ -316,17 +315,23 @@ void odb_add_to_alternates_file(struct object_database *odb,\n> > \tfree(alts);\n> > }\n> > \n> > -void odb_add_to_alternates_memory(struct object_database *odb,\n> > -\t\t\t\t const char *reference)\n> > +struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n> > +\t\t\t\t\t\tconst char *reference)\n> > {\n> > +\tstruct odb_source *alternate;\n> > +\tchar *objdir;\n> > +\n> > \t/*\n> > \t * Make sure alternates are initialized, or else our entry may be\n> > \t * overwritten when they are.\n> > \t */\n> > \todb_prepare_alternates(odb);\n> > \n> > -\tlink_alt_odb_entries(odb, reference,\n> > -\t\t\t '\\n', NULL, 0);\n> > +\tobjdir = real_pathdup(odb->sources->path, 1);\n> > +\talternate = link_alt_odb_entry(odb, reference, NULL, 0, objdir);\n> \n> If I understand correctly, instead of using real_pathdup() we could\n> instead call:\n> \n> alternate = link_alt_odb_entry(odb, reference, \"/\", 0, odb->sources->path);\n> \n> I did not test this, but it would avoid duplicating the path here. I'm\n> not sure though whether it's easier to read.\n\nWe can't quite, as the call to `alt_odb_usable()` needs the normalized\nobject directory path so that it can figure out whether its local object\ndirectory is the same as the proposed new object directory.\n\nBut what I noticed is that we're passing in redundant information: the\npath is already stored in `struct object_database`, and all callers of\n`link_alt_odb_entry()` pass both that ODB and its path. So we can drop\nthe parameter and make it an internal implementation detail.\n\nPatrick\n"},{"id":"523725","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH v2 0/9] midx: stop duplicating info redundant with their sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:50Z","receivedAt":"2025-08-07T08:10:05Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series is a follow up to the recent patch series that started\nto move the MIDX into its object source. It refactors the MIDX subsystem\nso that we always read and write them via an ODB source. This ultimately\nallows us to store a pointer to the owning ODB source in the MIDX and\nthus dedup some redundant information.\n\nThis series is built on top of e813a0200a7 (The fifteenth batch,\n2025-07-28) with ps/object-store-midx at ec865d94d46 (midx: remove\nnow-unused linked list of multi-pack indices, 2025-07-15) merged into\nit.\n\nChanges in v2:\n - Fix a comment typo.\n - Introduce another commit that simplifies the calling convention for\n `link_alt_odb_entry()`.\n - Link to v1: https://lore.kernel.org/r/20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (9):\n odb: store locality in object database sources\n odb: allow `odb_find_source()` to fail\n odb: return newly created in-memory sources\n odb: simplify calling `link_alt_odb_entry()`\n midx: drop redundant `struct repository` parameter\n midx: load multi-pack indices via their source\n midx: write multi-pack indices via their source\n midx: stop duplicating info redundant with its owning source\n midx: compute paths via their source\n\n builtin/commit-graph.c | 4 ++\n builtin/multi-pack-index.c | 31 +++++++---\n builtin/pack-objects.c | 2 +-\n builtin/repack.c | 7 ++-\n midx-write.c | 116 +++++++++++++++++--------------------\n midx.c | 135 +++++++++++++++++++++-----------------------\n midx.h | 42 ++++++--------\n odb.c | 43 ++++++--------\n odb.h | 16 ++++--\n pack-bitmap.c | 15 +++--\n pack-revindex.c | 14 ++---\n packfile.c | 13 ++---\n repository.c | 1 +\n t/helper/test-read-midx.c | 31 +++++-----\n t/t5319-multi-pack-index.sh | 8 +--\n 15 files changed, 239 insertions(+), 239 deletions(-)\n\nRange-diff versus v1:\n\n 1: 38e145e1e8 ! 1: d61e0702df odb: store locality in object database sources\n @@ odb.h: struct odb_source {\n \tstruct multi_pack_index *midx;\n \n +\t/*\n -+\t * Figure out whether this is the local alternate of the owning\n ++\t * Figure out whether this is the local source of the owning\n +\t * repository, which would typically be its \".git/objects\" directory.\n +\t * This local object directory is usually where objects would be\n +\t * written to.\n 2: bdf2e55514 = 2: 5f0a422b17 odb: allow `odb_find_source()` to fail\n 3: 6e11c5ecb5 = 3: 698f8793fd odb: return newly created in-memory sources\n -: ---------- > 4: 6fd1530502 odb: simplify calling `link_alt_odb_entry()`\n 4: 52b0e420d1 = 5: 90d9dbb3f7 midx: drop redundant `struct repository` parameter\n 5: b1c200817a = 6: 2c4d45d912 midx: load multi-pack indices via their source\n 6: 9b064ce6b2 = 7: d4d7ccab0e midx: write multi-pack indices via their source\n 7: 0553fb6b8a = 8: 9a6ce9872c midx: stop duplicating info redundant with its owning source\n 8: 13c480bac9 = 9: b814e2f328 midx: compute paths via their source\n\n---\nbase-commit: ee2fd3d4b224c8145eeff04101ea6075560973e1\nchange-id: 20250729-b4-pks-midx-deduplicate-source-info-fcf3c08bfd53\n\n"},{"id":"523726","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-1-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 1/9] odb: store locality in object database sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:51Z","receivedAt":"2025-08-07T08:10:05Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Object database sources are classified either as:\n\n - Local, which means that the source is the repository's primary\n source. This is typically \".git/objects\".\n\n - Non-local, which is everything else. Most importantly this includes\n alternates and quarantine directories.\n\nThis locality is often computed ad-hoc by checking whether a given\nobject source is the first one. This works, but it is quite roundabout.\n\nRefactor the code so that we store locality when creating the sources in\nthe first place. This makes it both more accessible and robust.\n\nSigned-off-by: Patrick Steinhardt \n---\n midx.c | 5 +++--\n midx.h | 2 +-\n odb.c | 1 +\n odb.h | 8 ++++++++\n packfile.c | 9 ++++-----\n repository.c | 1 +\n 6 files changed, 18 insertions(+), 8 deletions(-)\n\ndiff --git a/midx.c b/midx.c\nindex 7d407682e6..b9ca0915a6 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -723,7 +723,7 @@ int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id)\n \treturn 0;\n }\n \n-int prepare_multi_pack_index_one(struct odb_source *source, int local)\n+int prepare_multi_pack_index_one(struct odb_source *source)\n {\n \tstruct repository *r = source->odb->repo;\n \n@@ -734,7 +734,8 @@ int prepare_multi_pack_index_one(struct odb_source *source, int local)\n \tif (source->midx)\n \t\treturn 1;\n \n-\tsource->midx = load_multi_pack_index(r, source->path, local);\n+\tsource->midx = load_multi_pack_index(r, source->path,\n+\t\t\t\t\t source->local);\n \n \treturn !!source->midx;\n }\ndiff --git a/midx.h b/midx.h\nindex 076382de8a..28c426a823 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -122,7 +122,7 @@ int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pa\n int midx_contains_pack(struct multi_pack_index *m,\n \t\t const char *idx_or_pack_name);\n int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);\n-int prepare_multi_pack_index_one(struct odb_source *source, int local);\n+int prepare_multi_pack_index_one(struct odb_source *source);\n \n /*\n * Variant of write_midx_file which writes a MIDX containing only the packs\ndiff --git a/odb.c b/odb.c\nindex 1f48a0448e..1761a50840 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -176,6 +176,7 @@ static int link_alt_odb_entry(struct object_database *odb,\n \n \tCALLOC_ARRAY(alternate, 1);\n \talternate->odb = odb;\n+\talternate->local = false;\n \t/* pathbuf.buf is already in r->objects->source_by_path */\n \talternate->path = strbuf_detach(&pathbuf, NULL);\n \ndiff --git a/odb.h b/odb.h\nindex 09177bf430..f9300439ba 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -63,6 +63,14 @@ struct odb_source {\n \t */\n \tstruct multi_pack_index *midx;\n \n+\t/*\n+\t * Figure out whether this is the local source of the owning\n+\t * repository, which would typically be its \".git/objects\" directory.\n+\t * This local object directory is usually where objects would be\n+\t * written to.\n+\t */\n+\tbool local;\n+\n \t/*\n \t * This is a temporary object store created by the tmp_objdir\n \t * facility. Disable ref updates since the objects in the store\ndiff --git a/packfile.c b/packfile.c\nindex 5d73932f50..a38544b87b 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -935,14 +935,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,\n \t\treport_garbage(PACKDIR_FILE_GARBAGE, full_name);\n }\n \n-static void prepare_packed_git_one(struct odb_source *source, int local)\n+static void prepare_packed_git_one(struct odb_source *source)\n {\n \tstruct string_list garbage = STRING_LIST_INIT_DUP;\n \tstruct prepare_pack_data data = {\n \t\t.m = source->midx,\n \t\t.r = source->odb->repo,\n \t\t.garbage = &garbage,\n-\t\t.local = local,\n+\t\t.local = source->local,\n \t};\n \n \tfor_each_file_in_pack_dir(source->path, prepare_pack, &data);\n@@ -1037,9 +1037,8 @@ static void prepare_packed_git(struct repository *r)\n \n \todb_prepare_alternates(r->objects);\n \tfor (source = r->objects->sources; source; source = source->next) {\n-\t\tint local = (source == r->objects->sources);\n-\t\tprepare_multi_pack_index_one(source, local);\n-\t\tprepare_packed_git_one(source, local);\n+\t\tprepare_multi_pack_index_one(source);\n+\t\tprepare_packed_git_one(source);\n \t}\n \trearrange_packed_git(r);\n \ndiff --git a/repository.c b/repository.c\nindex ecd691181f..97f0578381 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -168,6 +168,7 @@ void repo_set_gitdir(struct repository *repo,\n \tif (!repo->objects->sources) {\n \t\tCALLOC_ARRAY(repo->objects->sources, 1);\n \t\trepo->objects->sources->odb = repo->objects;\n+\t\trepo->objects->sources->local = true;\n \t\trepo->objects->sources_tail = &repo->objects->sources->next;\n \t}\n \texpand_base_dir(&repo->objects->sources->path, o->object_dir,\n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523727","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-2-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 2/9] odb: allow `odb_find_source()` to fail","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:52Z","receivedAt":"2025-08-07T08:10:08Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When trying to locate a source for an unknown object directory we will\ndie right away. In subsequent patches we will add new callsites though\nthat want to handle this situation gracefully instead.\n\nRefactor the function to return a `NULL` pointer if the source could not\nbe found and adapt the callsites to die instead.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/commit-graph.c | 4 ++++\n midx-write.c | 2 ++\n odb.c | 2 --\n odb.h | 4 ++--\n 4 files changed, 8 insertions(+), 4 deletions(-)\n\ndiff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\nindex 25018a0b9d..dc2c1a5432 100644\n--- a/builtin/commit-graph.c\n+++ b/builtin/commit-graph.c\n@@ -102,6 +102,8 @@ static int graph_verify(int argc, const char **argv, const char *prefix,\n \t\tflags |= COMMIT_GRAPH_WRITE_PROGRESS;\n \n \tsource = odb_find_source(the_repository->objects, opts.obj_dir);\n+\tif (!source)\n+\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n \tgraph_name = get_commit_graph_filename(source);\n \tchain_name = get_commit_graph_chain_filename(source);\n \tif (open_commit_graph(graph_name, &fd, &st))\n@@ -290,6 +292,8 @@ static int graph_write(int argc, const char **argv, const char *prefix,\n \t\tflags |= COMMIT_GRAPH_WRITE_BLOOM_FILTERS;\n \n \tsource = odb_find_source(the_repository->objects, opts.obj_dir);\n+\tif (!source)\n+\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n \n \tif (opts.reachable) {\n \t\tif (write_commit_graph_reachable(source, flags, &write_opts))\ndiff --git a/midx-write.c b/midx-write.c\nindex c1ae62d354..40580d8c73 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -917,6 +917,8 @@ static struct multi_pack_index *lookup_multi_pack_index(struct repository *r,\n \t\t\t\t\t\t\tconst char *object_dir)\n {\n \tstruct odb_source *source = odb_find_source(r->objects, object_dir);\n+\tif (!source)\n+\t\tdie(_(\"could not find object directory matching %s\"), object_dir);\n \treturn get_multi_pack_index(source);\n }\n \ndiff --git a/odb.c b/odb.c\nindex 1761a50840..61104b7cb8 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -464,8 +464,6 @@ struct odb_source *odb_find_source(struct object_database *odb, const char *obj_\n \tfree(obj_dir_real);\n \tstrbuf_release(&odb_path_real);\n \n-\tif (!source)\n-\t\tdie(_(\"could not find object directory matching %s\"), obj_dir);\n \treturn source;\n }\n \ndiff --git a/odb.h b/odb.h\nindex f9300439ba..d085e691f2 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -186,8 +186,8 @@ struct object_database *odb_new(struct repository *repo);\n void odb_clear(struct object_database *o);\n \n /*\n- * Find source by its object directory path. Dies in case the source couldn't\n- * be found.\n+ * Find source by its object directory path. Returns a `NULL` pointer in case\n+ * the source could not be found.\n */\n struct odb_source *odb_find_source(struct object_database *odb, const char *obj_dir);\n \n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523728","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-3-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 3/9] odb: return newly created in-memory sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:53Z","receivedAt":"2025-08-07T08:10:12Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Callers have no trivial way to obtain the newly created object database\nsource when adding it to the in-memory list of alternates. While not yet\nneeded anywhere, a subsequent commit will want to obtain that pointer.\n\nRefactor the function to return the source to make it easily accessible.\n\nSigned-off-by: Patrick Steinhardt \n---\n odb.c | 37 +++++++++++++++++++++----------------\n odb.h | 4 ++--\n 2 files changed, 23 insertions(+), 18 deletions(-)\n\ndiff --git a/odb.c b/odb.c\nindex 61104b7cb8..7793816f81 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -139,23 +139,22 @@ static void read_info_alternates(struct object_database *odb,\n \t\t\t\t const char *relative_base,\n \t\t\t\t int depth);\n \n-static int link_alt_odb_entry(struct object_database *odb,\n-\t\t\t const struct strbuf *entry,\n-\t\t\t const char *relative_base,\n-\t\t\t int depth,\n-\t\t\t const char *normalized_objdir)\n+static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n+\t\t\t\t\t const char *entry,\n+\t\t\t\t\t const char *relative_base,\n+\t\t\t\t\t int depth,\n+\t\t\t\t\t const char *normalized_objdir)\n {\n-\tstruct odb_source *alternate;\n+\tstruct odb_source *alternate = NULL;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n \tstruct strbuf tmp = STRBUF_INIT;\n \tkhiter_t pos;\n-\tint ret = -1;\n \n-\tif (!is_absolute_path(entry->buf) && relative_base) {\n+\tif (!is_absolute_path(entry) && relative_base) {\n \t\tstrbuf_realpath(&pathbuf, relative_base, 1);\n \t\tstrbuf_addch(&pathbuf, '/');\n \t}\n-\tstrbuf_addbuf(&pathbuf, entry);\n+\tstrbuf_addstr(&pathbuf, entry);\n \n \tif (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {\n \t\terror(_(\"unable to normalize alternate object path: %s\"),\n@@ -189,11 +188,11 @@ static int link_alt_odb_entry(struct object_database *odb,\n \n \t/* recursively add alternates */\n \tread_info_alternates(odb, alternate->path, depth + 1);\n-\tret = 0;\n+\n error:\n \tstrbuf_release(&tmp);\n \tstrbuf_release(&pathbuf);\n-\treturn ret;\n+\treturn alternate;\n }\n \n static const char *parse_alt_odb_entry(const char *string,\n@@ -246,7 +245,7 @@ static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(odb, &entry,\n+\t\tlink_alt_odb_entry(odb, entry.buf,\n \t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n \tstrbuf_release(&entry);\n@@ -316,17 +315,23 @@ void odb_add_to_alternates_file(struct object_database *odb,\n \tfree(alts);\n }\n \n-void odb_add_to_alternates_memory(struct object_database *odb,\n-\t\t\t\t const char *reference)\n+struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n+\t\t\t\t\t\tconst char *reference)\n {\n+\tstruct odb_source *alternate;\n+\tchar *objdir;\n+\n \t/*\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n \todb_prepare_alternates(odb);\n \n-\tlink_alt_odb_entries(odb, reference,\n-\t\t\t '\\n', NULL, 0);\n+\tobjdir = real_pathdup(odb->sources->path, 1);\n+\talternate = link_alt_odb_entry(odb, reference, NULL, 0, objdir);\n+\n+\tfree(objdir);\n+\treturn alternate;\n }\n \n struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,\ndiff --git a/odb.h b/odb.h\nindex d085e691f2..aeca9d0a70 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -265,8 +265,8 @@ void odb_add_to_alternates_file(struct object_database *odb,\n * recursive alternates it points to), but do not modify the on-disk alternates\n * file.\n */\n-void odb_add_to_alternates_memory(struct object_database *odb,\n-\t\t\t\t const char *dir);\n+struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n+\t\t\t\t\t\tconst char *dir);\n \n /*\n * Read an object from the database. Returns the object data and assigns object\n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523729","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-4-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 4/9] odb: simplify calling `link_alt_odb_entry()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:54Z","receivedAt":"2025-08-07T08:10:15Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Callers of `link_alt_odb_entry()` are expected to pass in three\ndifferent paths:\n\n - The (potentially relative) path of the object directory that we're\n about to add.\n\n - The base that should be used to resolve a relative object directory\n path.\n\n - The resolved path to the object database's objects directory.\n\nJuggling those three paths makes the calling convention somewhat hard to\ngrok at first.\n\nAs it turns out, the third parameter is redundant: we always pass in the\nresolved path of the object database's primary source, and we already\npass in the database itself. So instead, we can resolve that path in the\nfunction itself.\n\nOne downside of this is that one caller of `link_alt_odb_entry()` calls\nthis function in a loop, so we were able to resolve the directory a\nsingle time, only. But ultimately, we only ever end up with a rather\nlimited number of alternates anyway, so the extra couple of cycles we\nsave feels more like a micro optimization.\n\nRefactor the code accordingly.\n\nSigned-off-by: Patrick Steinhardt \n---\n odb.c | 25 +++++++------------------\n 1 file changed, 7 insertions(+), 18 deletions(-)\n\ndiff --git a/odb.c b/odb.c\nindex 7793816f81..7c28462c41 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -142,8 +142,7 @@ static void read_info_alternates(struct object_database *odb,\n static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n \t\t\t\t\t const char *entry,\n \t\t\t\t\t const char *relative_base,\n-\t\t\t\t\t int depth,\n-\t\t\t\t\t const char *normalized_objdir)\n+\t\t\t\t\t int depth)\n {\n \tstruct odb_source *alternate = NULL;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n@@ -170,7 +169,10 @@ static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(odb, &pathbuf, normalized_objdir, &pos))\n+\tstrbuf_reset(&tmp);\n+\tstrbuf_realpath(&tmp, odb->sources->path, 1);\n+\n+\tif (!alt_odb_usable(odb, &pathbuf, tmp.buf, &pos))\n \t\tgoto error;\n \n \tCALLOC_ARRAY(alternate, 1);\n@@ -227,7 +229,6 @@ static const char *parse_alt_odb_entry(const char *string,\n static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n \t\t\t\t int sep, const char *relative_base, int depth)\n {\n-\tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf entry = STRBUF_INIT;\n \n \tif (!alt || !*alt)\n@@ -239,17 +240,13 @@ static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n \t\treturn;\n \t}\n \n-\tstrbuf_realpath(&objdirbuf, odb->sources->path, 1);\n-\n \twhile (*alt) {\n \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n \t\tif (!entry.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(odb, entry.buf,\n-\t\t\t\t relative_base, depth, objdirbuf.buf);\n+\t\tlink_alt_odb_entry(odb, entry.buf, relative_base, depth);\n \t}\n \tstrbuf_release(&entry);\n-\tstrbuf_release(&objdirbuf);\n }\n \n static void read_info_alternates(struct object_database *odb,\n@@ -318,20 +315,12 @@ void odb_add_to_alternates_file(struct object_database *odb,\n struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n \t\t\t\t\t\tconst char *reference)\n {\n-\tstruct odb_source *alternate;\n-\tchar *objdir;\n-\n \t/*\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n \todb_prepare_alternates(odb);\n-\n-\tobjdir = real_pathdup(odb->sources->path, 1);\n-\talternate = link_alt_odb_entry(odb, reference, NULL, 0, objdir);\n-\n-\tfree(objdir);\n-\treturn alternate;\n+\treturn link_alt_odb_entry(odb, reference, NULL, 0);\n }\n \n struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,\n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523730","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-5-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 5/9] midx: drop redundant `struct repository` parameter","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:55Z","receivedAt":"2025-08-07T08:10:19Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"There are a couple of functions that take both a `struct repository` and\na `struct multi_pack_index`. This provides redundant information though\nwithout much benefit given that the multi-pack index already has a\npointer to its owning repository.\n\nDrop the `struct repository` parameter from such functions. While at it,\nreorder the list of parameters of `fill_midx_entry()` so that the MIDX\ncomes first to better align with our coding guidelines.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/pack-objects.c | 2 +-\n midx-write.c | 16 +++++++---------\n midx.c | 18 +++++++++---------\n midx.h | 6 +++---\n pack-bitmap.c | 4 ++--\n packfile.c | 4 ++--\n t/helper/test-read-midx.c | 4 ++--\n 7 files changed, 26 insertions(+), 28 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 3dd84495b8..b9fd685b8f 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1733,7 +1733,7 @@ static int want_object_in_pack_mtime(const struct object_id *oid,\n \t\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \t\tstruct pack_entry e;\n \n-\t\tif (m && fill_midx_entry(the_repository, oid, &e, m)) {\n+\t\tif (m && fill_midx_entry(m, oid, &e)) {\n \t\t\twant = want_object_in_pack_one(e.p, oid, exclude, found_pack, found_offset, found_mtime);\n \t\t\tif (want != -1)\n \t\t\t\treturn want;\ndiff --git a/midx-write.c b/midx-write.c\nindex 40580d8c73..37a0b1682f 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -944,8 +944,7 @@ static int fill_packs_from_midx(struct write_midx_context *ctx,\n \t\t\t */\n \t\t\tif (flags & MIDX_WRITE_REV_INDEX ||\n \t\t\t preferred_pack_name) {\n-\t\t\t\tif (prepare_midx_pack(ctx->repo, m,\n-\t\t\t\t\t\t m->num_packs_in_base + i)) {\n+\t\t\t\tif (prepare_midx_pack(m, m->num_packs_in_base + i)) {\n \t\t\t\t\terror(_(\"could not load pack\"));\n \t\t\t\t\treturn 1;\n \t\t\t\t}\n@@ -1568,7 +1567,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \t\tif (count[i])\n \t\t\tcontinue;\n \n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tcontinue;\n \n \t\tif (m->packs[i]->pack_keep || m->packs[i]->is_cruft)\n@@ -1614,13 +1613,12 @@ static int compare_by_mtime(const void *a_, const void *b_)\n \treturn 0;\n }\n \n-static int want_included_pack(struct repository *r,\n-\t\t\t struct multi_pack_index *m,\n+static int want_included_pack(struct multi_pack_index *m,\n \t\t\t int pack_kept_objects,\n \t\t\t uint32_t pack_int_id)\n {\n \tstruct packed_git *p;\n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn 0;\n \tp = m->packs[pack_int_id];\n \tif (!pack_kept_objects && p->pack_keep)\n@@ -1642,7 +1640,7 @@ static void fill_included_packs_all(struct repository *r,\n \trepo_config_get_bool(r, \"repack.packkeptobjects\", &pack_kept_objects);\n \n \tfor (i = 0; i < m->num_packs; i++) {\n-\t\tif (!want_included_pack(r, m, pack_kept_objects, i))\n+\t\tif (!want_included_pack(m, pack_kept_objects, i))\n \t\t\tcontinue;\n \n \t\tinclude_pack[i] = 1;\n@@ -1666,7 +1664,7 @@ static void fill_included_packs_batch(struct repository *r,\n \tfor (i = 0; i < m->num_packs; i++) {\n \t\tpack_info[i].pack_int_id = i;\n \n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tcontinue;\n \n \t\tpack_info[i].mtime = m->packs[i]->mtime;\n@@ -1685,7 +1683,7 @@ static void fill_included_packs_batch(struct repository *r,\n \t\tstruct packed_git *p = m->packs[pack_int_id];\n \t\tuint64_t expected_size;\n \n-\t\tif (!want_included_pack(r, m, pack_kept_objects, pack_int_id))\n+\t\tif (!want_included_pack(m, pack_kept_objects, pack_int_id))\n \t\t\tcontinue;\n \n \t\t/*\ndiff --git a/midx.c b/midx.c\nindex b9ca0915a6..8459dda8c9 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -450,9 +450,10 @@ static uint32_t midx_for_pack(struct multi_pack_index **_m,\n \treturn pack_int_id - m->num_packs_in_base;\n }\n \n-int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,\n+int prepare_midx_pack(struct multi_pack_index *m,\n \t\t uint32_t pack_int_id)\n {\n+\tstruct repository *r = m->repo;\n \tstruct strbuf pack_name = STRBUF_INIT;\n \tstruct strbuf key = STRBUF_INIT;\n \tstruct packed_git *p;\n@@ -507,7 +508,7 @@ struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \n #define MIDX_CHUNK_BITMAPPED_PACKS_WIDTH (2 * sizeof(uint32_t))\n \n-int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n+int nth_bitmapped_pack(struct multi_pack_index *m,\n \t\t struct bitmapped_pack *bp, uint32_t pack_int_id)\n {\n \tuint32_t local_pack_int_id = midx_for_pack(&m, pack_int_id);\n@@ -515,7 +516,7 @@ int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n \tif (!m->chunk_bitmapped_packs)\n \t\treturn error(_(\"MIDX does not contain the BTMP chunk\"));\n \n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn error(_(\"could not load bitmapped pack %\"PRIu32), pack_int_id);\n \n \tbp->p = m->packs[local_pack_int_id];\n@@ -600,10 +601,9 @@ uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos)\n \t\t\t\t\t (off_t)pos * MIDX_CHUNK_OFFSET_WIDTH);\n }\n \n-int fill_midx_entry(struct repository *r,\n+int fill_midx_entry(struct multi_pack_index *m,\n \t\t const struct object_id *oid,\n-\t\t struct pack_entry *e,\n-\t\t struct multi_pack_index *m)\n+\t\t struct pack_entry *e)\n {\n \tuint32_t pos;\n \tuint32_t pack_int_id;\n@@ -615,7 +615,7 @@ int fill_midx_entry(struct repository *r,\n \tmidx_for_object(&m, pos);\n \tpack_int_id = nth_midxed_pack_int_id(m, pos);\n \n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn 0;\n \tp = m->packs[pack_int_id - m->num_packs_in_base];\n \n@@ -912,7 +912,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \t\t\t\t\t\t _(\"Looking for referenced packfiles\"),\n \t\t\t\t\t\t m->num_packs + m->num_packs_in_base);\n \tfor (i = 0; i < m->num_packs + m->num_packs_in_base; i++) {\n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tmidx_report(\"failed to load pack in position %d\", i);\n \n \t\tdisplay_progress(progress, i + 1);\n@@ -989,7 +989,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \n \t\tnth_midxed_object_oid(&oid, m, pairs[i].pos);\n \n-\t\tif (!fill_midx_entry(r, &oid, &e, m)) {\n+\t\tif (!fill_midx_entry(m, &oid, &e)) {\n \t\t\tmidx_report(_(\"failed to load pack entry for oid[%d] = %s\"),\n \t\t\t\t pairs[i].pos, oid_to_hex(&oid));\n \t\t\tcontinue;\ndiff --git a/midx.h b/midx.h\nindex 28c426a823..f7e07083e1 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -103,10 +103,10 @@ void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n struct multi_pack_index *load_multi_pack_index(struct repository *r,\n \t\t\t\t\t const char *object_dir,\n \t\t\t\t\t int local);\n-int prepare_midx_pack(struct repository *r, struct multi_pack_index *m, uint32_t pack_int_id);\n+int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);\n struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \t\t\t\t uint32_t pack_int_id);\n-int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n+int nth_bitmapped_pack(struct multi_pack_index *m,\n \t\t struct bitmapped_pack *bp, uint32_t pack_int_id);\n int bsearch_one_midx(const struct object_id *oid, struct multi_pack_index *m,\n \t\t uint32_t *result);\n@@ -118,7 +118,7 @@ uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos);\n struct object_id *nth_midxed_object_oid(struct object_id *oid,\n \t\t\t\t\tstruct multi_pack_index *m,\n \t\t\t\t\tuint32_t n);\n-int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pack_entry *e, struct multi_pack_index *m);\n+int fill_midx_entry(struct multi_pack_index *m, const struct object_id *oid, struct pack_entry *e);\n int midx_contains_pack(struct multi_pack_index *m,\n \t\t const char *idx_or_pack_name);\n int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex d14421ee20..fb0b11ca07 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -493,7 +493,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t}\n \n \tfor (i = 0; i < bitmap_git->midx->num_packs + bitmap_git->midx->num_packs_in_base; i++) {\n-\t\tif (prepare_midx_pack(bitmap_repo(bitmap_git), bitmap_git->midx, i)) {\n+\t\tif (prepare_midx_pack(bitmap_git->midx, i)) {\n \t\t\twarning(_(\"could not open pack %s\"),\n \t\t\t\tbitmap_git->midx->pack_names[i]);\n \t\t\tgoto cleanup;\n@@ -2466,7 +2466,7 @@ void reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git,\n \t\tstruct multi_pack_index *m = bitmap_git->midx;\n \t\tfor (i = 0; i < m->num_packs + m->num_packs_in_base; i++) {\n \t\t\tstruct bitmapped_pack pack;\n-\t\t\tif (nth_bitmapped_pack(r, bitmap_git->midx, &pack, i) < 0) {\n+\t\t\tif (nth_bitmapped_pack(bitmap_git->midx, &pack, i) < 0) {\n \t\t\t\twarning(_(\"unable to load pack: '%s', disabling pack-reuse\"),\n \t\t\t\t\tbitmap_git->midx->pack_names[i]);\n \t\t\t\tfree(packs);\ndiff --git a/packfile.c b/packfile.c\nindex a38544b87b..acb680966d 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -1091,7 +1091,7 @@ struct packed_git *get_all_packs(struct repository *r)\n \t\tif (!m)\n \t\t\tcontinue;\n \t\tfor (uint32_t i = 0; i < m->num_packs + m->num_packs_in_base; i++)\n-\t\t\tprepare_midx_pack(r, m, i);\n+\t\t\tprepare_midx_pack(m, i);\n \t}\n \n \treturn r->objects->packed_git;\n@@ -2077,7 +2077,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa\n \tprepare_packed_git(r);\n \n \tfor (struct odb_source *source = r->objects->sources; source; source = source->next)\n-\t\tif (source->midx && fill_midx_entry(r, oid, e, source->midx))\n+\t\tif (source->midx && fill_midx_entry(source->midx, oid, e))\n \t\t\treturn 1;\n \n \tif (!r->objects->packed_git)\ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex da2aa036b5..e430aa247c 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -65,7 +65,7 @@ static int read_midx_file(const char *object_dir, const char *checksum,\n \t\tfor (i = 0; i < m->num_objects; i++) {\n \t\t\tnth_midxed_object_oid(&oid, m,\n \t\t\t\t\t i + m->num_objects_in_base);\n-\t\t\tfill_midx_entry(the_repository, &oid, &e, m);\n+\t\t\tfill_midx_entry(m, &oid, &e);\n \n \t\t\tprintf(\"%s %\"PRIu64\"\\t%s\\n\",\n \t\t\t oid_to_hex(&oid), e.offset, e.p->pack_name);\n@@ -126,7 +126,7 @@ static int read_midx_bitmapped_packs(const char *object_dir)\n \t\treturn 1;\n \n \tfor (i = 0; i < midx->num_packs + midx->num_packs_in_base; i++) {\n-\t\tif (nth_bitmapped_pack(the_repository, midx, &pack, i) < 0) {\n+\t\tif (nth_bitmapped_pack(midx, &pack, i) < 0) {\n \t\t\tclose_midx(midx);\n \t\t\treturn 1;\n \t\t}\n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523731","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-6-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 6/9] midx: load multi-pack indices via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:56Z","receivedAt":"2025-08-07T08:10:22Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"To load a multi-pack index the caller is expected to pass both the\nrepository and the object directory where the multi-pack index is\nlocated. While this works, this layout has a couple of downsides:\n\n - We need to pass in information reduntant with the owning source,\n namely its object directory and whether the source is local or not.\n\n - We don't have access to the source when loading the multi-pack\n index. If we had that access, we could store a pointer to the owning\n source in the MIDX and thus deduplicate some information.\n\n - Multi-pack indices are inherently specific to the object source and\n its format. With the goal of pluggable object backends in mind we\n will eventually want the backends to own the logic of reading and\n writing multi-pack indices. Making the logic work on top of object\n sources is a step into that direction.\n\nRefactor loading of multi-pack indices accordingly.\n\nThis surfaces one small problem though: git-multi-pack-index(1) and our\nMIDX test helper both know to read and write multi-pack-indices located\nin a different object directory. This issue is addressed by adding the\nuser-provided object directory as an in-memory alternate.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/multi-pack-index.c | 18 ++++++++++++--\n midx.c | 57 ++++++++++++++++++++-------------------------\n midx.h | 6 ++---\n t/helper/test-read-midx.c | 25 ++++++++++++--------\n t/t5319-multi-pack-index.sh | 8 +++----\n 5 files changed, 62 insertions(+), 52 deletions(-)\n\ndiff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c\nindex aa25b06f9d..e4a9305af3 100644\n--- a/builtin/multi-pack-index.c\n+++ b/builtin/multi-pack-index.c\n@@ -64,12 +64,20 @@ static int parse_object_dir(const struct option *opt, const char *arg,\n \tchar **value = opt->value;\n \tfree(*value);\n \tif (unset)\n-\t\t*value = xstrdup(repo_get_object_directory(the_repository));\n+\t\t*value = xstrdup(the_repository->objects->sources->path);\n \telse\n \t\t*value = real_pathdup(arg, 1);\n \treturn 0;\n }\n \n+static struct odb_source *handle_object_dir_option(struct repository *repo)\n+{\n+\tstruct odb_source *source = odb_find_source(repo->objects, opts.object_dir);\n+\tif (!source)\n+\t\tsource = odb_add_to_alternates_memory(repo->objects, opts.object_dir);\n+\treturn source;\n+}\n+\n static struct option common_opts[] = {\n \tOPT_CALLBACK(0, \"object-dir\", &opts.object_dir,\n \t N_(\"directory\"),\n@@ -157,6 +165,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_write_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(repo);\n \n \tFREE_AND_NULL(options);\n \n@@ -193,6 +202,8 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,\n \t\t\tN_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n+\n \toptions = add_common_options(builtin_multi_pack_index_verify_options);\n \n \ttrace2_cmd_mode(argv[0]);\n@@ -205,10 +216,11 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_verify_usage,\n \t\t\t\t options);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn verify_midx_file(the_repository, opts.object_dir, opts.flags);\n+\treturn verify_midx_file(source, opts.flags);\n }\n \n static int cmd_multi_pack_index_expire(int argc, const char **argv,\n@@ -233,6 +245,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_expire_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n@@ -265,6 +278,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_repack_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \ndiff --git a/midx.c b/midx.c\nindex 8459dda8c9..831a7e9b5f 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -95,11 +95,10 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,\n \treturn 0;\n }\n \n-static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,\n-\t\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t\t const char *midx_name,\n-\t\t\t\t\t\t\t int local)\n+static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,\n+\t\t\t\t\t\t\t const char *midx_name)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct multi_pack_index *m = NULL;\n \tint fd;\n \tstruct stat st;\n@@ -129,10 +128,10 @@ static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,\n \tmidx_map = xmmap(NULL, midx_size, PROT_READ, MAP_PRIVATE, fd, 0);\n \tclose(fd);\n \n-\tFLEX_ALLOC_STR(m, object_dir, object_dir);\n+\tFLEX_ALLOC_STR(m, object_dir, source->path);\n \tm->data = midx_map;\n \tm->data_len = midx_size;\n-\tm->local = local;\n+\tm->local = source->local;\n \tm->repo = r;\n \n \tm->signature = get_be32(m->data);\n@@ -297,19 +296,18 @@ static int add_midx_to_chain(struct multi_pack_index *midx,\n \treturn 1;\n }\n \n-static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n-\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t int local,\n+static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source *source,\n \t\t\t\t\t\t int fd, struct stat *st,\n \t\t\t\t\t\t int *incomplete_chain)\n {\n+\tconst struct git_hash_algo *hash_algo = source->odb->repo->hash_algo;\n \tstruct multi_pack_index *midx_chain = NULL;\n \tstruct strbuf buf = STRBUF_INIT;\n \tint valid = 1;\n \tuint32_t i, count;\n \tFILE *fp = xfdopen(fd, \"r\");\n \n-\tcount = st->st_size / (r->hash_algo->hexsz + 1);\n+\tcount = st->st_size / (hash_algo->hexsz + 1);\n \n \tfor (i = 0; i < count; i++) {\n \t\tstruct multi_pack_index *m;\n@@ -318,7 +316,7 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \t\tif (strbuf_getline_lf(&buf, fp) == EOF)\n \t\t\tbreak;\n \n-\t\tif (get_oid_hex_algop(buf.buf, &layer, r->hash_algo)) {\n+\t\tif (get_oid_hex_algop(buf.buf, &layer, hash_algo)) {\n \t\t\twarning(_(\"invalid multi-pack-index chain: line '%s' \"\n \t\t\t\t \"not a hash\"),\n \t\t\t\tbuf.buf);\n@@ -329,9 +327,9 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \t\tvalid = 0;\n \n \t\tstrbuf_reset(&buf);\n-\t\tget_split_midx_filename_ext(r->hash_algo, &buf, object_dir,\n+\t\tget_split_midx_filename_ext(hash_algo, &buf, source->path,\n \t\t\t\t\t layer.hash, MIDX_EXT_MIDX);\n-\t\tm = load_multi_pack_index_one(r, object_dir, buf.buf, local);\n+\t\tm = load_multi_pack_index_one(source, buf.buf);\n \n \t\tif (m) {\n \t\t\tif (add_midx_to_chain(m, midx_chain)) {\n@@ -354,40 +352,35 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \treturn midx_chain;\n }\n \n-static struct multi_pack_index *load_multi_pack_index_chain(struct repository *r,\n-\t\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t\t int local)\n+static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *source)\n {\n \tstruct strbuf chain_file = STRBUF_INIT;\n \tstruct stat st;\n \tint fd;\n \tstruct multi_pack_index *m = NULL;\n \n-\tget_midx_chain_filename(&chain_file, object_dir);\n-\tif (open_multi_pack_index_chain(r->hash_algo, chain_file.buf, &fd, &st)) {\n+\tget_midx_chain_filename(&chain_file, source->path);\n+\tif (open_multi_pack_index_chain(source->odb->repo->hash_algo, chain_file.buf, &fd, &st)) {\n \t\tint incomplete;\n \t\t/* ownership of fd is taken over by load function */\n-\t\tm = load_midx_chain_fd_st(r, object_dir, local, fd, &st,\n-\t\t\t\t\t &incomplete);\n+\t\tm = load_midx_chain_fd_st(source, fd, &st, &incomplete);\n \t}\n \n \tstrbuf_release(&chain_file);\n \treturn m;\n }\n \n-struct multi_pack_index *load_multi_pack_index(struct repository *r,\n-\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t int local)\n+struct multi_pack_index *load_multi_pack_index(struct odb_source *source)\n {\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tstruct multi_pack_index *m;\n \n-\tget_midx_filename(r->hash_algo, &midx_name, object_dir);\n+\tget_midx_filename(source->odb->repo->hash_algo, &midx_name,\n+\t\t\t source->path);\n \n-\tm = load_multi_pack_index_one(r, object_dir,\n-\t\t\t\t midx_name.buf, local);\n+\tm = load_multi_pack_index_one(source, midx_name.buf);\n \tif (!m)\n-\t\tm = load_multi_pack_index_chain(r, object_dir, local);\n+\t\tm = load_multi_pack_index_chain(source);\n \n \tstrbuf_release(&midx_name);\n \n@@ -734,8 +727,7 @@ int prepare_multi_pack_index_one(struct odb_source *source)\n \tif (source->midx)\n \t\treturn 1;\n \n-\tsource->midx = load_multi_pack_index(r, source->path,\n-\t\t\t\t\t source->local);\n+\tsource->midx = load_multi_pack_index(source);\n \n \treturn !!source->midx;\n }\n@@ -880,12 +872,13 @@ static int compare_pair_pos_vs_id(const void *_a, const void *_b)\n \t\t\tdisplay_progress(progress, _n); \\\n \t} while (0)\n \n-int verify_midx_file(struct repository *r, const char *object_dir, unsigned flags)\n+int verify_midx_file(struct odb_source *source, unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct pair_pos_vs_id *pairs = NULL;\n \tuint32_t i;\n \tstruct progress *progress = NULL;\n-\tstruct multi_pack_index *m = load_multi_pack_index(r, object_dir, 1);\n+\tstruct multi_pack_index *m = load_multi_pack_index(source);\n \tstruct multi_pack_index *curr;\n \tverify_midx_error = 0;\n \n@@ -894,7 +887,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \t\tstruct stat sb;\n \t\tstruct strbuf filename = STRBUF_INIT;\n \n-\t\tget_midx_filename(r->hash_algo, &filename, object_dir);\n+\t\tget_midx_filename(r->hash_algo, &filename, source->path);\n \n \t\tif (!stat(filename.buf, &sb)) {\n \t\t\terror(_(\"multi-pack-index file exists, but failed to parse\"));\ndiff --git a/midx.h b/midx.h\nindex f7e07083e1..970d043989 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -100,9 +100,7 @@ void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n \t\t\t\t struct strbuf *buf, const char *object_dir,\n \t\t\t\t const unsigned char *hash, const char *ext);\n \n-struct multi_pack_index *load_multi_pack_index(struct repository *r,\n-\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t int local);\n+struct multi_pack_index *load_multi_pack_index(struct odb_source *source);\n int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);\n struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \t\t\t\t uint32_t pack_int_id);\n@@ -136,7 +134,7 @@ int write_midx_file_only(struct repository *r, const char *object_dir,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags);\n void clear_midx_file(struct repository *r);\n-int verify_midx_file(struct repository *r, const char *object_dir, unsigned flags);\n+int verify_midx_file(struct odb_source *source, unsigned flags);\n int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags);\n int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags);\n \ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex e430aa247c..bcb8ea7671 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -11,14 +11,24 @@\n #include \"gettext.h\"\n #include \"pack-revindex.h\"\n \n+static struct multi_pack_index *setup_midx(const char *object_dir)\n+{\n+\tstruct odb_source *source;\n+\tsetup_git_directory();\n+\tsource = odb_find_source(the_repository->objects, object_dir);\n+\tif (!source)\n+\t\tsource = odb_add_to_alternates_memory(the_repository->objects,\n+\t\t\t\t\t\t object_dir);\n+\treturn load_multi_pack_index(source);\n+}\n+\n static int read_midx_file(const char *object_dir, const char *checksum,\n \t\t\t int show_objects)\n {\n \tuint32_t i;\n \tstruct multi_pack_index *m;\n \n-\tsetup_git_directory();\n-\tm = load_multi_pack_index(the_repository, object_dir, 1);\n+\tm = setup_midx(object_dir);\n \n \tif (!m)\n \t\treturn 1;\n@@ -81,8 +91,7 @@ static int read_midx_checksum(const char *object_dir)\n {\n \tstruct multi_pack_index *m;\n \n-\tsetup_git_directory();\n-\tm = load_multi_pack_index(the_repository, object_dir, 1);\n+\tm = setup_midx(object_dir);\n \tif (!m)\n \t\treturn 1;\n \tprintf(\"%s\\n\", hash_to_hex(get_midx_checksum(m)));\n@@ -96,9 +105,7 @@ static int read_midx_preferred_pack(const char *object_dir)\n \tstruct multi_pack_index *midx = NULL;\n \tuint32_t preferred_pack;\n \n-\tsetup_git_directory();\n-\n-\tmidx = load_multi_pack_index(the_repository, object_dir, 1);\n+\tmidx = setup_midx(object_dir);\n \tif (!midx)\n \t\treturn 1;\n \n@@ -119,9 +126,7 @@ static int read_midx_bitmapped_packs(const char *object_dir)\n \tstruct bitmapped_pack pack;\n \tuint32_t i;\n \n-\tsetup_git_directory();\n-\n-\tmidx = load_multi_pack_index(the_repository, object_dir, 1);\n+\tmidx = setup_midx(object_dir);\n \tif (!midx)\n \t\treturn 1;\n \ndiff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh\nindex bd75dea950..4e5e882989 100755\n--- a/t/t5319-multi-pack-index.sh\n+++ b/t/t5319-multi-pack-index.sh\n@@ -28,11 +28,11 @@ midx_read_expect () {\n \t\tEOF\n \t\tif test $NUM_PACKS -ge 1\n \t\tthen\n-\t\t\tls $OBJECT_DIR/pack/ | grep idx | sort\n+\t\t\tls \"$OBJECT_DIR\"/pack/ | grep idx | sort\n \t\tfi &&\n \t\tprintf \"object-dir: $OBJECT_DIR\\n\"\n \t} >expect &&\n-\ttest-tool read-midx $OBJECT_DIR >actual &&\n+\ttest-tool read-midx \"$OBJECT_DIR\" >actual &&\n \ttest_cmp expect actual\n }\n \n@@ -305,7 +305,7 @@ test_expect_success 'midx picks objects from preferred pack' '\n \n \t\tofs=$(git show-index expect &&\n \t\tgrep ^$b out >actual &&\n \n@@ -639,7 +639,7 @@ test_expect_success 'force some 64-bit offsets with pack-objects' '\n \t\t( cd ../objects64 && pwd ) >.git/objects/info/alternates &&\n \t\tmidx64=$(git multi-pack-index --object-dir=../objects64 write)\n \t) &&\n-\tmidx_read_expect 1 63 5 objects64 \" large-offsets\"\n+\tmidx_read_expect 1 63 5 \"$(pwd)/objects64\" \" large-offsets\"\n '\n \n test_expect_success 'verify multi-pack-index with 64-bit offsets' '\n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523732","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-7-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 7/9] midx: write multi-pack indices via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:57Z","receivedAt":"2025-08-07T08:10:25Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Similar to the preceding commit, refactor the writing side of multi-pack\nindices so that we pass in the object database source where the index\nshould be written to.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/multi-pack-index.c | 19 +++++++------\n builtin/repack.c | 2 +-\n midx-write.c | 69 +++++++++++++++++++++-------------------------\n midx.h | 8 +++---\n 4 files changed, 47 insertions(+), 51 deletions(-)\n\ndiff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c\nindex e4a9305af3..b1e971e535 100644\n--- a/builtin/multi-pack-index.c\n+++ b/builtin/multi-pack-index.c\n@@ -147,6 +147,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \t\t\t N_(\"refs snapshot for selecting bitmap commits\")),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n \tint ret;\n \n \topts.flags |= MIDX_WRITE_BITMAP_HASH_CACHE;\n@@ -165,7 +166,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_write_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(repo);\n+\tsource = handle_object_dir_option(repo);\n \n \tFREE_AND_NULL(options);\n \n@@ -174,7 +175,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \n \t\tread_packs_from_stdin(&packs);\n \n-\t\tret = write_midx_file_only(repo, opts.object_dir, &packs,\n+\t\tret = write_midx_file_only(source, &packs,\n \t\t\t\t\t opts.preferred_pack,\n \t\t\t\t\t opts.refs_snapshot, opts.flags);\n \n@@ -185,7 +186,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \n \t}\n \n-\tret = write_midx_file(repo, opts.object_dir, opts.preferred_pack,\n+\tret = write_midx_file(source, opts.preferred_pack,\n \t\t\t opts.refs_snapshot, opts.flags);\n \n \tfree(opts.refs_snapshot);\n@@ -233,6 +234,8 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \t\t\tN_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n+\n \toptions = add_common_options(builtin_multi_pack_index_expire_options);\n \n \ttrace2_cmd_mode(argv[0]);\n@@ -245,11 +248,11 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_expire_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(the_repository);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn expire_midx_packs(the_repository, opts.object_dir, opts.flags);\n+\treturn expire_midx_packs(source, opts.flags);\n }\n \n static int cmd_multi_pack_index_repack(int argc, const char **argv,\n@@ -264,6 +267,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \t\t N_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n \n \toptions = add_common_options(builtin_multi_pack_index_repack_options);\n \n@@ -278,12 +282,11 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_repack_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(the_repository);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn midx_repack(the_repository, opts.object_dir,\n-\t\t\t (size_t)opts.batch_size, opts.flags);\n+\treturn midx_repack(source, (size_t)opts.batch_size, opts.flags);\n }\n \n int cmd_multi_pack_index(int argc,\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 21723866b9..94dec26f18 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -1711,7 +1711,7 @@ int cmd_repack(int argc,\n \t\tunsigned flags = 0;\n \t\tif (git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL, 0))\n \t\t\tflags |= MIDX_WRITE_INCREMENTAL;\n-\t\twrite_midx_file(the_repository, repo_get_object_directory(the_repository),\n+\t\twrite_midx_file(the_repository->objects->sources,\n \t\t\t\tNULL, NULL, flags);\n \t}\n \ndiff --git a/midx-write.c b/midx-write.c\nindex 37a0b1682f..bf7c01d4b1 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -913,15 +913,6 @@ static int write_midx_bitmap(struct write_midx_context *ctx,\n \treturn ret;\n }\n \n-static struct multi_pack_index *lookup_multi_pack_index(struct repository *r,\n-\t\t\t\t\t\t\tconst char *object_dir)\n-{\n-\tstruct odb_source *source = odb_find_source(r->objects, object_dir);\n-\tif (!source)\n-\t\tdie(_(\"could not find object directory matching %s\"), object_dir);\n-\treturn get_multi_pack_index(source);\n-}\n-\n static int fill_packs_from_midx(struct write_midx_context *ctx,\n \t\t\t\tconst char *preferred_pack_name, uint32_t flags)\n {\n@@ -1012,7 +1003,7 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \treturn ret;\n }\n \n-static void clear_midx_files(struct repository *r, const char *object_dir,\n+static void clear_midx_files(struct odb_source *source,\n \t\t\t const char **hashes, uint32_t hashes_nr,\n \t\t\t unsigned incremental)\n {\n@@ -1031,16 +1022,16 @@ static void clear_midx_files(struct repository *r, const char *object_dir,\n \tuint32_t i, j;\n \n \tfor (i = 0; i < ARRAY_SIZE(exts); i++) {\n-\t\tclear_incremental_midx_files_ext(object_dir, exts[i],\n+\t\tclear_incremental_midx_files_ext(source->path, exts[i],\n \t\t\t\t\t\t hashes, hashes_nr);\n \t\tfor (j = 0; j < hashes_nr; j++)\n-\t\t\tclear_midx_files_ext(object_dir, exts[i], hashes[j]);\n+\t\t\tclear_midx_files_ext(source->path, exts[i], hashes[j]);\n \t}\n \n \tif (incremental)\n-\t\tget_midx_filename(r->hash_algo, &buf, object_dir);\n+\t\tget_midx_filename(source->odb->repo->hash_algo, &buf, source->path);\n \telse\n-\t\tget_midx_chain_filename(&buf, object_dir);\n+\t\tget_midx_chain_filename(&buf, source->path);\n \n \tif (unlink(buf.buf) && errno != ENOENT)\n \t\tdie_errno(_(\"failed to clear multi-pack-index at %s\"), buf.buf);\n@@ -1048,13 +1039,14 @@ static void clear_midx_files(struct repository *r, const char *object_dir,\n \tstrbuf_release(&buf);\n }\n \n-static int write_midx_internal(struct repository *r, const char *object_dir,\n+static int write_midx_internal(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t struct string_list *packs_to_drop,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot,\n \t\t\t unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tunsigned char midx_hash[GIT_MAX_RAWSZ];\n \tuint32_t i, start_pack;\n@@ -1078,15 +1070,15 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (ctx.incremental)\n \t\tstrbuf_addf(&midx_name,\n \t\t\t \"%s/pack/multi-pack-index.d/tmp_midx_XXXXXX\",\n-\t\t\t object_dir);\n+\t\t\t source->path);\n \telse\n-\t\tget_midx_filename(r->hash_algo, &midx_name, object_dir);\n+\t\tget_midx_filename(r->hash_algo, &midx_name, source->path);\n \tif (safe_create_leading_directories(r, midx_name.buf))\n \t\tdie_errno(_(\"unable to create leading directories of %s\"),\n \t\t\t midx_name.buf);\n \n \tif (!packs_to_include || ctx.incremental) {\n-\t\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\t\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \t\tif (m && !midx_checksum_valid(m)) {\n \t\t\twarning(_(\"ignoring existing multi-pack-index; checksum mismatch\"));\n \t\t\tm = NULL;\n@@ -1140,7 +1132,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \n \tctx.to_include = packs_to_include;\n \n-\tfor_each_file_in_pack_dir(object_dir, add_pack_to_midx, &ctx);\n+\tfor_each_file_in_pack_dir(source->path, add_pack_to_midx, &ctx);\n \tstop_progress(&ctx.progress);\n \n \tif ((ctx.m && ctx.nr == ctx.m->num_packs + ctx.m->num_packs_in_base) &&\n@@ -1160,7 +1152,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\t\t * corresponding bitmap (or one wasn't requested).\n \t\t\t */\n \t\t\tif (!want_bitmap)\n-\t\t\t\tclear_midx_files_ext(object_dir, \"bitmap\", NULL);\n+\t\t\t\tclear_midx_files_ext(source->path, \"bitmap\", NULL);\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -1328,7 +1320,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (ctx.incremental) {\n \t\tstruct strbuf lock_name = STRBUF_INIT;\n \n-\t\tget_midx_chain_filename(&lock_name, object_dir);\n+\t\tget_midx_chain_filename(&lock_name, source->path);\n \t\thold_lock_file_for_update(&lk, lock_name.buf, LOCK_DIE_ON_ERROR);\n \t\tstrbuf_release(&lock_name);\n \n@@ -1391,7 +1383,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \n \tif (flags & MIDX_WRITE_REV_INDEX &&\n \t git_env_bool(\"GIT_TEST_MIDX_WRITE_REV\", 0))\n-\t\twrite_midx_reverse_index(&ctx, object_dir, midx_hash);\n+\t\twrite_midx_reverse_index(&ctx, source->path, midx_hash);\n \n \tif (flags & MIDX_WRITE_BITMAP) {\n \t\tstruct packing_data pdata;\n@@ -1414,7 +1406,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\tFREE_AND_NULL(ctx.entries);\n \t\tctx.entries_nr = 0;\n \n-\t\tif (write_midx_bitmap(&ctx, object_dir,\n+\t\tif (write_midx_bitmap(&ctx, source->path,\n \t\t\t\t midx_hash, &pdata, commits, commits_nr,\n \t\t\t\t flags) < 0) {\n \t\t\terror(_(\"could not write multi-pack bitmap\"));\n@@ -1448,7 +1440,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\t\treturn -1;\n \n \t\tget_split_midx_filename_ext(r->hash_algo, &final_midx_name,\n-\t\t\t\t\t object_dir, midx_hash, MIDX_EXT_MIDX);\n+\t\t\t\t\t source->path, midx_hash, MIDX_EXT_MIDX);\n \n \t\tif (rename_tempfile(&incr, final_midx_name.buf) < 0) {\n \t\t\terror_errno(_(\"unable to rename new multi-pack-index layer\"));\n@@ -1481,7 +1473,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (commit_lock_file(&lk) < 0)\n \t\tdie_errno(_(\"could not write multi-pack-index\"));\n \n-\tclear_midx_files(r, object_dir, keep_hashes,\n+\tclear_midx_files(source, keep_hashes,\n \t\t\t ctx.num_multi_pack_indexes_before + 1,\n \t\t\t ctx.incremental);\n \n@@ -1510,29 +1502,29 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \treturn result;\n }\n \n-int write_midx_file(struct repository *r, const char *object_dir,\n+int write_midx_file(struct odb_source *source,\n \t\t const char *preferred_pack_name,\n \t\t const char *refs_snapshot, unsigned flags)\n {\n-\treturn write_midx_internal(r, object_dir, NULL, NULL,\n+\treturn write_midx_internal(source, NULL, NULL,\n \t\t\t\t preferred_pack_name, refs_snapshot,\n \t\t\t\t flags);\n }\n \n-int write_midx_file_only(struct repository *r, const char *object_dir,\n+int write_midx_file_only(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags)\n {\n-\treturn write_midx_internal(r, object_dir, packs_to_include, NULL,\n+\treturn write_midx_internal(source, packs_to_include, NULL,\n \t\t\t\t preferred_pack_name, refs_snapshot, flags);\n }\n \n-int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags)\n+int expire_midx_packs(struct odb_source *source, unsigned flags)\n {\n \tuint32_t i, *count, result = 0;\n \tstruct string_list packs_to_drop = STRING_LIST_INIT_DUP;\n-\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \tstruct progress *progress = NULL;\n \n \tif (!m)\n@@ -1545,7 +1537,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \n \tif (flags & MIDX_PROGRESS)\n \t\tprogress = start_delayed_progress(\n-\t\t\t\t\t r,\n+\t\t\t\t\t source->odb->repo,\n \t\t\t\t\t _(\"Counting referenced objects\"),\n \t\t\t\t\t m->num_objects);\n \tfor (i = 0; i < m->num_objects; i++) {\n@@ -1557,7 +1549,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \n \tif (flags & MIDX_PROGRESS)\n \t\tprogress = start_delayed_progress(\n-\t\t\t\t\t r,\n+\t\t\t\t\t source->odb->repo,\n \t\t\t\t\t _(\"Finding and deleting unreferenced packfiles\"),\n \t\t\t\t\t m->num_packs);\n \tfor (i = 0; i < m->num_packs; i++) {\n@@ -1585,7 +1577,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \tfree(count);\n \n \tif (packs_to_drop.nr)\n-\t\tresult = write_midx_internal(r, object_dir, NULL,\n+\t\tresult = write_midx_internal(source, NULL,\n \t\t\t\t\t &packs_to_drop, NULL, NULL, flags);\n \n \tstring_list_clear(&packs_to_drop, 0);\n@@ -1710,14 +1702,15 @@ static void fill_included_packs_batch(struct repository *r,\n \tfree(pack_info);\n }\n \n-int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags)\n+int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tint result = 0;\n \tuint32_t i, packs_to_repack = 0;\n \tunsigned char *include_pack;\n \tstruct child_process cmd = CHILD_PROCESS_INIT;\n \tFILE *cmd_in;\n-\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \n \t/*\n \t * When updating the default for these configuration\n@@ -1751,7 +1744,7 @@ int midx_repack(struct repository *r, const char *object_dir, size_t batch_size,\n \n \tstrvec_push(&cmd.args, \"pack-objects\");\n \n-\tstrvec_pushf(&cmd.args, \"%s/pack/pack\", object_dir);\n+\tstrvec_pushf(&cmd.args, \"%s/pack/pack\", source->path);\n \n \tif (delta_base_offset)\n \t\tstrvec_push(&cmd.args, \"--delta-base-offset\");\n@@ -1792,7 +1785,7 @@ int midx_repack(struct repository *r, const char *object_dir, size_t batch_size,\n \t\tgoto cleanup;\n \t}\n \n-\tresult = write_midx_internal(r, object_dir, NULL, NULL, NULL, NULL,\n+\tresult = write_midx_internal(source, NULL, NULL, NULL, NULL,\n \t\t\t\t flags);\n \n cleanup:\ndiff --git a/midx.h b/midx.h\nindex 970d043989..d162001fbb 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -126,17 +126,17 @@ int prepare_multi_pack_index_one(struct odb_source *source);\n * Variant of write_midx_file which writes a MIDX containing only the packs\n * specified in packs_to_include.\n */\n-int write_midx_file(struct repository *r, const char *object_dir,\n+int write_midx_file(struct odb_source *source,\n \t\t const char *preferred_pack_name, const char *refs_snapshot,\n \t\t unsigned flags);\n-int write_midx_file_only(struct repository *r, const char *object_dir,\n+int write_midx_file_only(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags);\n void clear_midx_file(struct repository *r);\n int verify_midx_file(struct odb_source *source, unsigned flags);\n-int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags);\n-int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags);\n+int expire_midx_packs(struct odb_source *source, unsigned flags);\n+int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags);\n \n void close_midx(struct multi_pack_index *m);\n \n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523733","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-8-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 8/9] midx: stop duplicating info redundant with its owning source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:58Z","receivedAt":"2025-08-07T08:10:28Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Multi-pack indices store some information that is redundant with their\nowning source:\n\n - The locality bit that tracks whether the source is the primary\n object source or an alternate.\n\n - The object directory path the multi-pack index is located in.\n\n - The pointer to the owning parent directory.\n\nAll of this information is already contained in `struct odb_source`. So\nnow that we always have that struct available when loading a multi-pack\nindex we have it readily accessible.\n\nDrop the redundant information and instead store a pointer to the object\nsource.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/repack.c | 5 +++--\n midx-write.c | 9 +++++----\n midx.c | 21 +++++++++++----------\n midx.h | 7 ++-----\n pack-bitmap.c | 13 +++++++------\n pack-revindex.c | 14 +++++++-------\n t/helper/test-read-midx.c | 2 +-\n 7 files changed, 36 insertions(+), 35 deletions(-)\n\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 94dec26f18..5af3e27357 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -223,9 +223,10 @@ static void mark_packs_for_deletion(struct existing_packs *existing,\n static void remove_redundant_pack(const char *dir_name, const char *base_name)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct multi_pack_index *m = get_multi_pack_index(the_repository->objects->sources);\n+\tstruct odb_source *source = the_repository->objects->sources;\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \tstrbuf_addf(&buf, \"%s.pack\", base_name);\n-\tif (m && m->local && midx_contains_pack(m, buf.buf))\n+\tif (m && source->local && midx_contains_pack(m, buf.buf))\n \t\tclear_midx_file(the_repository);\n \tstrbuf_insertf(&buf, 0, \"%s/\", dir_name);\n \tunlink_pack_path(buf.buf, 1);\ndiff --git a/midx-write.c b/midx-write.c\nindex bf7c01d4b1..84f76856d6 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -981,10 +981,11 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \tfor (i = 0; i < ARRAY_SIZE(midx_exts); i++) {\n \t\tconst unsigned char *hash = get_midx_checksum(m);\n \n-\t\tget_midx_filename_ext(m->repo->hash_algo, &from, m->object_dir,\n+\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &from,\n+\t\t\t\t m->source->path,\n \t\t\t\t hash, midx_exts[i].non_split);\n-\t\tget_split_midx_filename_ext(m->repo->hash_algo, &to,\n-\t\t\t\t\t m->object_dir, hash,\n+\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &to,\n+\t\t\t\t\t m->source->path, hash,\n \t\t\t\t\t midx_exts[i].split);\n \n \t\tif (link(from.buf, to.buf) < 0 && errno != ENOENT) {\n@@ -1109,7 +1110,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\tif (flags & MIDX_WRITE_BITMAP && load_midx_revindex(m)) {\n \t\t\t\terror(_(\"could not load reverse index for MIDX %s\"),\n \t\t\t\t hash_to_hex_algop(get_midx_checksum(m),\n-\t\t\t\t\t\t\tm->repo->hash_algo));\n+\t\t\t\t\t\t\tm->source->odb->repo->hash_algo));\n \t\t\t\tresult = 1;\n \t\t\t\tgoto cleanup;\n \t\t\t}\ndiff --git a/midx.c b/midx.c\nindex 831a7e9b5f..81bf3c4d5f 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -26,7 +26,7 @@ int cmp_idx_or_pack_name(const char *idx_or_pack_name,\n \n const unsigned char *get_midx_checksum(struct multi_pack_index *m)\n {\n-\treturn m->data + m->data_len - m->repo->hash_algo->rawsz;\n+\treturn m->data + m->data_len - m->source->odb->repo->hash_algo->rawsz;\n }\n \n void get_midx_filename(const struct git_hash_algo *hash_algo,\n@@ -128,11 +128,10 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou\n \tmidx_map = xmmap(NULL, midx_size, PROT_READ, MAP_PRIVATE, fd, 0);\n \tclose(fd);\n \n-\tFLEX_ALLOC_STR(m, object_dir, source->path);\n+\tCALLOC_ARRAY(m, 1);\n \tm->data = midx_map;\n \tm->data_len = midx_size;\n-\tm->local = source->local;\n-\tm->repo = r;\n+\tm->source = source;\n \n \tm->signature = get_be32(m->data);\n \tif (m->signature != MIDX_SIGNATURE)\n@@ -446,7 +445,7 @@ static uint32_t midx_for_pack(struct multi_pack_index **_m,\n int prepare_midx_pack(struct multi_pack_index *m,\n \t\t uint32_t pack_int_id)\n {\n-\tstruct repository *r = m->repo;\n+\tstruct repository *r = m->source->odb->repo;\n \tstruct strbuf pack_name = STRBUF_INIT;\n \tstruct strbuf key = STRBUF_INIT;\n \tstruct packed_git *p;\n@@ -458,7 +457,7 @@ int prepare_midx_pack(struct multi_pack_index *m,\n \tif (m->packs[pack_int_id])\n \t\treturn 0;\n \n-\tstrbuf_addf(&pack_name, \"%s/pack/%s\", m->object_dir,\n+\tstrbuf_addf(&pack_name, \"%s/pack/%s\", m->source->path,\n \t\t m->pack_names[pack_int_id]);\n \n \t/* pack_map holds the \".pack\" name, but we have the .idx */\n@@ -469,7 +468,8 @@ int prepare_midx_pack(struct multi_pack_index *m,\n \t\t\t\t\tstrhash(key.buf), key.buf,\n \t\t\t\t\tstruct packed_git, packmap_ent);\n \tif (!p) {\n-\t\tp = add_packed_git(r, pack_name.buf, pack_name.len, m->local);\n+\t\tp = add_packed_git(r, pack_name.buf, pack_name.len,\n+\t\t\t\t m->source->local);\n \t\tif (p) {\n \t\t\tinstall_packed_git(r, p);\n \t\t\tlist_add_tail(&p->mru, &r->objects->packed_git_mru);\n@@ -528,7 +528,8 @@ int bsearch_one_midx(const struct object_id *oid, struct multi_pack_index *m,\n \t\t uint32_t *result)\n {\n \tint ret = bsearch_hash(oid->hash, m->chunk_oid_fanout,\n-\t\t\t m->chunk_oid_lookup, m->repo->hash_algo->rawsz,\n+\t\t\t m->chunk_oid_lookup,\n+\t\t\t m->source->odb->repo->hash_algo->rawsz,\n \t\t\t result);\n \tif (result)\n \t\t*result += m->num_objects_in_base;\n@@ -559,7 +560,7 @@ struct object_id *nth_midxed_object_oid(struct object_id *oid,\n \tn = midx_for_object(&m, n);\n \n \toidread(oid, m->chunk_oid_lookup + st_mult(m->hash_len, n),\n-\t\tm->repo->hash_algo);\n+\t\tm->source->odb->repo->hash_algo);\n \treturn oid;\n }\n \n@@ -734,7 +735,7 @@ int prepare_multi_pack_index_one(struct odb_source *source)\n \n int midx_checksum_valid(struct multi_pack_index *m)\n {\n-\treturn hashfile_checksum_valid(m->repo->hash_algo,\n+\treturn hashfile_checksum_valid(m->source->odb->repo->hash_algo,\n \t\t\t\t m->data, m->data_len);\n }\n \ndiff --git a/midx.h b/midx.h\nindex d162001fbb..71dbdec66e 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -35,6 +35,8 @@ struct odb_source;\n \t\"GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL\"\n \n struct multi_pack_index {\n+\tstruct odb_source *source;\n+\n \tconst unsigned char *data;\n \tsize_t data_len;\n \n@@ -50,7 +52,6 @@ struct multi_pack_index {\n \tuint32_t num_objects;\n \tint preferred_pack_idx;\n \n-\tint local;\n \tint has_chain;\n \n \tconst unsigned char *chunk_pack_names;\n@@ -71,10 +72,6 @@ struct multi_pack_index {\n \n \tconst char **pack_names;\n \tstruct packed_git **packs;\n-\n-\tstruct repository *repo;\n-\n-\tchar object_dir[FLEX_ARRAY];\n };\n \n #define MIDX_PROGRESS (1 << 0)\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex fb0b11ca07..01e14c34bd 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -216,7 +216,7 @@ static uint32_t bitmap_num_objects(struct bitmap_index *index)\n static struct repository *bitmap_repo(struct bitmap_index *bitmap_git)\n {\n \tif (bitmap_is_midx(bitmap_git))\n-\t\treturn bitmap_git->midx->repo;\n+\t\treturn bitmap_git->midx->source->odb->repo;\n \treturn bitmap_git->pack->repo;\n }\n \n@@ -418,13 +418,13 @@ char *midx_bitmap_filename(struct multi_pack_index *midx)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n \tif (midx->has_chain)\n-\t\tget_split_midx_filename_ext(midx->repo->hash_algo, &buf,\n-\t\t\t\t\t midx->object_dir,\n+\t\tget_split_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t\t midx->source->path,\n \t\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t\t MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(midx->repo->hash_algo, &buf,\n-\t\t\t\t midx->object_dir, get_midx_checksum(midx),\n+\t\tget_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t midx->source->path, get_midx_checksum(midx),\n \t\t\t\t MIDX_EXT_BITMAP);\n \n \treturn strbuf_detach(&buf, NULL);\n@@ -463,7 +463,8 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tget_midx_filename(midx->repo->hash_algo, &buf, midx->object_dir);\n+\t\tget_midx_filename(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t midx->source->path);\n \t\ttrace2_data_string(\"bitmap\", bitmap_repo(bitmap_git),\n \t\t\t\t \"ignoring extra midx bitmap file\", buf.buf);\n \t\tclose(fd);\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 0cc422a1e6..b206518dcb 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -379,25 +379,25 @@ int load_midx_revindex(struct multi_pack_index *m)\n \t\t * not want to accidentally call munmap() in the middle of the\n \t\t * MIDX.\n \t\t */\n-\t\ttrace2_data_string(\"load_midx_revindex\", m->repo,\n+\t\ttrace2_data_string(\"load_midx_revindex\", m->source->odb->repo,\n \t\t\t\t \"source\", \"midx\");\n \t\tm->revindex_data = (const uint32_t *)m->chunk_revindex;\n \t\treturn 0;\n \t}\n \n-\ttrace2_data_string(\"load_midx_revindex\", m->repo,\n+\ttrace2_data_string(\"load_midx_revindex\", m->source->odb->repo,\n \t\t\t \"source\", \"rev\");\n \n \tif (m->has_chain)\n-\t\tget_split_midx_filename_ext(m->repo->hash_algo, &revindex_name,\n-\t\t\t\t\t m->object_dir, get_midx_checksum(m),\n+\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n+\t\t\t\t\t m->source->path, get_midx_checksum(m),\n \t\t\t\t\t MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(m->repo->hash_algo, &revindex_name,\n-\t\t\t\t m->object_dir, get_midx_checksum(m),\n+\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n+\t\t\t\t m->source->path, get_midx_checksum(m),\n \t\t\t\t MIDX_EXT_REV);\n \n-\tret = load_revindex_from_disk(m->repo->hash_algo,\n+\tret = load_revindex_from_disk(m->source->odb->repo->hash_algo,\n \t\t\t\t revindex_name.buf,\n \t\t\t\t m->num_objects,\n \t\t\t\t &m->revindex_map,\ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex bcb8ea7671..6de5d1665a 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -66,7 +66,7 @@ static int read_midx_file(const char *object_dir, const char *checksum,\n \tfor (i = 0; i < m->num_packs; i++)\n \t\tprintf(\"%s\\n\", m->pack_names[i]);\n \n-\tprintf(\"object-dir: %s\\n\", m->object_dir);\n+\tprintf(\"object-dir: %s\\n\", m->source->path);\n \n \tif (show_objects) {\n \t\tstruct object_id oid;\n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523734","messageId":"20250807-b4-pks-midx-deduplicate-source-info-v2-9-bcffb8fc119c@pks.im","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"[PATCH v2 9/9] midx: compute paths via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-07T08:09:59Z","receivedAt":"2025-08-07T08:10:31Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"With the preceding commits we started to always have the object database\nsource available when we load, write or access multi-pack indices. With\nthis in place we can change how MIDX paths are computed so that we don't\nhave to pass in the combination of a hash algorithm and object directory\nanymore, but only the object database source.\n\nRefactor the code accordingly.\n\nSigned-off-by: Patrick Steinhardt \n---\n midx-write.c | 52 ++++++++++++++++++++++++----------------------------\n midx.c | 54 +++++++++++++++++++++++++-----------------------------\n midx.h | 13 +++++--------\n pack-bitmap.c | 10 ++++------\n pack-revindex.c | 8 ++++----\n 5 files changed, 62 insertions(+), 75 deletions(-)\n\ndiff --git a/midx-write.c b/midx-write.c\nindex 84f76856d6..1dcdf3dc0f 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -26,9 +26,9 @@\n #define MIDX_CHUNK_LARGE_OFFSET_WIDTH (sizeof(uint64_t))\n \n extern int midx_checksum_valid(struct multi_pack_index *m);\n-extern void clear_midx_files_ext(const char *object_dir, const char *ext,\n+extern void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t const char *keep_hash);\n-extern void clear_incremental_midx_files_ext(const char *object_dir,\n+extern void clear_incremental_midx_files_ext(struct odb_source *source,\n \t\t\t\t\t const char *ext,\n \t\t\t\t\t const char **keep_hashes,\n \t\t\t\t\t uint32_t hashes_nr);\n@@ -112,6 +112,7 @@ struct write_midx_context {\n \tstruct string_list *to_include;\n \n \tstruct repository *repo;\n+\tstruct odb_source *source;\n };\n \n static int should_include_pack(const struct write_midx_context *ctx,\n@@ -648,7 +649,6 @@ static uint32_t *midx_pack_order(struct write_midx_context *ctx)\n }\n \n static void write_midx_reverse_index(struct write_midx_context *ctx,\n-\t\t\t\t const char *object_dir,\n \t\t\t\t unsigned char *midx_hash)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -657,11 +657,10 @@ static void write_midx_reverse_index(struct write_midx_context *ctx,\n \ttrace2_region_enter(\"midx\", \"write_midx_reverse_index\", ctx->repo);\n \n \tif (ctx->incremental)\n-\t\tget_split_midx_filename_ext(ctx->repo->hash_algo, &buf,\n-\t\t\t\t\t object_dir, midx_hash,\n-\t\t\t\t\t MIDX_EXT_REV);\n+\t\tget_split_midx_filename_ext(ctx->source, &buf,\n+\t\t\t\t\t midx_hash, MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(ctx->repo->hash_algo, &buf, object_dir,\n+\t\tget_midx_filename_ext(ctx->source, &buf,\n \t\t\t\t midx_hash, MIDX_EXT_REV);\n \n \ttmp_file = write_rev_file_order(ctx->repo, NULL, ctx->pack_order,\n@@ -836,7 +835,6 @@ static struct commit **find_commits_for_midx_bitmap(uint32_t *indexed_commits_nr\n }\n \n static int write_midx_bitmap(struct write_midx_context *ctx,\n-\t\t\t const char *object_dir,\n \t\t\t const unsigned char *midx_hash,\n \t\t\t struct packing_data *pdata,\n \t\t\t struct commit **commits,\n@@ -852,12 +850,11 @@ static int write_midx_bitmap(struct write_midx_context *ctx,\n \ttrace2_region_enter(\"midx\", \"write_midx_bitmap\", ctx->repo);\n \n \tif (ctx->incremental)\n-\t\tget_split_midx_filename_ext(ctx->repo->hash_algo, &bitmap_name,\n-\t\t\t\t\t object_dir, midx_hash,\n-\t\t\t\t\t MIDX_EXT_BITMAP);\n+\t\tget_split_midx_filename_ext(ctx->source, &bitmap_name,\n+\t\t\t\t\t midx_hash, MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(ctx->repo->hash_algo, &bitmap_name,\n-\t\t\t\t object_dir, midx_hash, MIDX_EXT_BITMAP);\n+\t\tget_midx_filename_ext(ctx->source, &bitmap_name,\n+\t\t\t\t midx_hash, MIDX_EXT_BITMAP);\n \n \tif (flags & MIDX_WRITE_BITMAP_HASH_CACHE)\n \t\toptions |= BITMAP_OPT_HASH_CACHE;\n@@ -981,11 +978,9 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \tfor (i = 0; i < ARRAY_SIZE(midx_exts); i++) {\n \t\tconst unsigned char *hash = get_midx_checksum(m);\n \n-\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &from,\n-\t\t\t\t m->source->path,\n+\t\tget_midx_filename_ext(m->source, &from,\n \t\t\t\t hash, midx_exts[i].non_split);\n-\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &to,\n-\t\t\t\t\t m->source->path, hash,\n+\t\tget_split_midx_filename_ext(m->source, &to, hash,\n \t\t\t\t\t midx_exts[i].split);\n \n \t\tif (link(from.buf, to.buf) < 0 && errno != ENOENT) {\n@@ -1023,16 +1018,16 @@ static void clear_midx_files(struct odb_source *source,\n \tuint32_t i, j;\n \n \tfor (i = 0; i < ARRAY_SIZE(exts); i++) {\n-\t\tclear_incremental_midx_files_ext(source->path, exts[i],\n+\t\tclear_incremental_midx_files_ext(source, exts[i],\n \t\t\t\t\t\t hashes, hashes_nr);\n \t\tfor (j = 0; j < hashes_nr; j++)\n-\t\t\tclear_midx_files_ext(source->path, exts[i], hashes[j]);\n+\t\t\tclear_midx_files_ext(source, exts[i], hashes[j]);\n \t}\n \n \tif (incremental)\n-\t\tget_midx_filename(source->odb->repo->hash_algo, &buf, source->path);\n+\t\tget_midx_filename(source, &buf);\n \telse\n-\t\tget_midx_chain_filename(&buf, source->path);\n+\t\tget_midx_chain_filename(source, &buf);\n \n \tif (unlink(buf.buf) && errno != ENOENT)\n \t\tdie_errno(_(\"failed to clear multi-pack-index at %s\"), buf.buf);\n@@ -1065,6 +1060,7 @@ static int write_midx_internal(struct odb_source *source,\n \ttrace2_region_enter(\"midx\", \"write_midx_internal\", r);\n \n \tctx.repo = r;\n+\tctx.source = source;\n \n \tctx.incremental = !!(flags & MIDX_WRITE_INCREMENTAL);\n \n@@ -1073,7 +1069,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\t \"%s/pack/multi-pack-index.d/tmp_midx_XXXXXX\",\n \t\t\t source->path);\n \telse\n-\t\tget_midx_filename(r->hash_algo, &midx_name, source->path);\n+\t\tget_midx_filename(source, &midx_name);\n \tif (safe_create_leading_directories(r, midx_name.buf))\n \t\tdie_errno(_(\"unable to create leading directories of %s\"),\n \t\t\t midx_name.buf);\n@@ -1153,7 +1149,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\t * corresponding bitmap (or one wasn't requested).\n \t\t\t */\n \t\t\tif (!want_bitmap)\n-\t\t\t\tclear_midx_files_ext(source->path, \"bitmap\", NULL);\n+\t\t\t\tclear_midx_files_ext(source, \"bitmap\", NULL);\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -1321,7 +1317,7 @@ static int write_midx_internal(struct odb_source *source,\n \tif (ctx.incremental) {\n \t\tstruct strbuf lock_name = STRBUF_INIT;\n \n-\t\tget_midx_chain_filename(&lock_name, source->path);\n+\t\tget_midx_chain_filename(source, &lock_name);\n \t\thold_lock_file_for_update(&lk, lock_name.buf, LOCK_DIE_ON_ERROR);\n \t\tstrbuf_release(&lock_name);\n \n@@ -1384,7 +1380,7 @@ static int write_midx_internal(struct odb_source *source,\n \n \tif (flags & MIDX_WRITE_REV_INDEX &&\n \t git_env_bool(\"GIT_TEST_MIDX_WRITE_REV\", 0))\n-\t\twrite_midx_reverse_index(&ctx, source->path, midx_hash);\n+\t\twrite_midx_reverse_index(&ctx, midx_hash);\n \n \tif (flags & MIDX_WRITE_BITMAP) {\n \t\tstruct packing_data pdata;\n@@ -1407,7 +1403,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\tFREE_AND_NULL(ctx.entries);\n \t\tctx.entries_nr = 0;\n \n-\t\tif (write_midx_bitmap(&ctx, source->path,\n+\t\tif (write_midx_bitmap(&ctx,\n \t\t\t\t midx_hash, &pdata, commits, commits_nr,\n \t\t\t\t flags) < 0) {\n \t\t\terror(_(\"could not write multi-pack bitmap\"));\n@@ -1440,8 +1436,8 @@ static int write_midx_internal(struct odb_source *source,\n \t\tif (link_midx_to_chain(ctx.base_midx) < 0)\n \t\t\treturn -1;\n \n-\t\tget_split_midx_filename_ext(r->hash_algo, &final_midx_name,\n-\t\t\t\t\t source->path, midx_hash, MIDX_EXT_MIDX);\n+\t\tget_split_midx_filename_ext(source, &final_midx_name,\n+\t\t\t\t\t midx_hash, MIDX_EXT_MIDX);\n \n \t\tif (rename_tempfile(&incr, final_midx_name.buf) < 0) {\n \t\t\terror_errno(_(\"unable to rename new multi-pack-index layer\"));\ndiff --git a/midx.c b/midx.c\nindex 81bf3c4d5f..7726c13d7e 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -16,9 +16,9 @@\n #define MIDX_PACK_ERROR ((void *)(intptr_t)-1)\n \n int midx_checksum_valid(struct multi_pack_index *m);\n-void clear_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t const char *keep_hash);\n-void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t char **keep_hashes,\n \t\t\t\t uint32_t hashes_nr);\n int cmp_idx_or_pack_name(const char *idx_or_pack_name,\n@@ -29,19 +29,17 @@ const unsigned char *get_midx_checksum(struct multi_pack_index *m)\n \treturn m->data + m->data_len - m->source->odb->repo->hash_algo->rawsz;\n }\n \n-void get_midx_filename(const struct git_hash_algo *hash_algo,\n-\t\t struct strbuf *out, const char *object_dir)\n+void get_midx_filename(struct odb_source *source, struct strbuf *out)\n {\n-\tget_midx_filename_ext(hash_algo, out, object_dir, NULL, NULL);\n+\tget_midx_filename_ext(source, out, NULL, NULL);\n }\n \n-void get_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t struct strbuf *out, const char *object_dir,\n+void get_midx_filename_ext(struct odb_source *source, struct strbuf *out,\n \t\t\t const unsigned char *hash, const char *ext)\n {\n-\tstrbuf_addf(out, \"%s/pack/multi-pack-index\", object_dir);\n+\tstrbuf_addf(out, \"%s/pack/multi-pack-index\", source->path);\n \tif (ext)\n-\t\tstrbuf_addf(out, \"-%s.%s\", hash_to_hex_algop(hash, hash_algo), ext);\n+\t\tstrbuf_addf(out, \"-%s.%s\", hash_to_hex_algop(hash, source->odb->repo->hash_algo), ext);\n }\n \n static int midx_read_oid_fanout(const unsigned char *chunk_start,\n@@ -222,24 +220,23 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou\n \treturn NULL;\n }\n \n-void get_midx_chain_dirname(struct strbuf *buf, const char *object_dir)\n+void get_midx_chain_dirname(struct odb_source *source, struct strbuf *buf)\n {\n-\tstrbuf_addf(buf, \"%s/pack/multi-pack-index.d\", object_dir);\n+\tstrbuf_addf(buf, \"%s/pack/multi-pack-index.d\", source->path);\n }\n \n-void get_midx_chain_filename(struct strbuf *buf, const char *object_dir)\n+void get_midx_chain_filename(struct odb_source *source, struct strbuf *buf)\n {\n-\tget_midx_chain_dirname(buf, object_dir);\n+\tget_midx_chain_dirname(source, buf);\n \tstrbuf_addstr(buf, \"/multi-pack-index-chain\");\n }\n \n-void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t\t struct strbuf *buf, const char *object_dir,\n+void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,\n \t\t\t\t const unsigned char *hash, const char *ext)\n {\n-\tget_midx_chain_dirname(buf, object_dir);\n+\tget_midx_chain_dirname(source, buf);\n \tstrbuf_addf(buf, \"/multi-pack-index-%s.%s\",\n-\t\t hash_to_hex_algop(hash, hash_algo), ext);\n+\t\t hash_to_hex_algop(hash, source->odb->repo->hash_algo), ext);\n }\n \n static int open_multi_pack_index_chain(const struct git_hash_algo *hash_algo,\n@@ -326,7 +323,7 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source *source,\n \t\tvalid = 0;\n \n \t\tstrbuf_reset(&buf);\n-\t\tget_split_midx_filename_ext(hash_algo, &buf, source->path,\n+\t\tget_split_midx_filename_ext(source, &buf,\n \t\t\t\t\t layer.hash, MIDX_EXT_MIDX);\n \t\tm = load_multi_pack_index_one(source, buf.buf);\n \n@@ -358,7 +355,7 @@ static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *s\n \tint fd;\n \tstruct multi_pack_index *m = NULL;\n \n-\tget_midx_chain_filename(&chain_file, source->path);\n+\tget_midx_chain_filename(source, &chain_file);\n \tif (open_multi_pack_index_chain(source->odb->repo->hash_algo, chain_file.buf, &fd, &st)) {\n \t\tint incomplete;\n \t\t/* ownership of fd is taken over by load function */\n@@ -374,8 +371,7 @@ struct multi_pack_index *load_multi_pack_index(struct odb_source *source)\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tstruct multi_pack_index *m;\n \n-\tget_midx_filename(source->odb->repo->hash_algo, &midx_name,\n-\t\t\t source->path);\n+\tget_midx_filename(source, &midx_name);\n \n \tm = load_multi_pack_index_one(source, midx_name.buf);\n \tif (!m)\n@@ -762,7 +758,7 @@ static void clear_midx_file_ext(const char *full_path, size_t full_path_len UNUS\n \t\tdie_errno(_(\"failed to remove %s\"), full_path);\n }\n \n-void clear_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t const char *keep_hash)\n {\n \tstruct clear_midx_data data;\n@@ -776,7 +772,7 @@ void clear_midx_files_ext(const char *object_dir, const char *ext,\n \t}\n \tdata.ext = ext;\n \n-\tfor_each_file_in_pack_dir(object_dir,\n+\tfor_each_file_in_pack_dir(source->path,\n \t\t\t\t clear_midx_file_ext,\n \t\t\t\t &data);\n \n@@ -785,7 +781,7 @@ void clear_midx_files_ext(const char *object_dir, const char *ext,\n \tfree(data.keep);\n }\n \n-void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t char **keep_hashes,\n \t\t\t\t uint32_t hashes_nr)\n {\n@@ -801,7 +797,7 @@ void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n \tdata.keep_nr = hashes_nr;\n \tdata.ext = ext;\n \n-\tfor_each_file_in_pack_subdir(object_dir, \"multi-pack-index.d\",\n+\tfor_each_file_in_pack_subdir(source->path, \"multi-pack-index.d\",\n \t\t\t\t clear_midx_file_ext, &data);\n \n \tfor (i = 0; i < hashes_nr; i++)\n@@ -813,7 +809,7 @@ void clear_midx_file(struct repository *r)\n {\n \tstruct strbuf midx = STRBUF_INIT;\n \n-\tget_midx_filename(r->hash_algo, &midx, r->objects->sources->path);\n+\tget_midx_filename(r->objects->sources, &midx);\n \n \tif (r->objects) {\n \t\tstruct odb_source *source;\n@@ -828,8 +824,8 @@ void clear_midx_file(struct repository *r)\n \tif (remove_path(midx.buf))\n \t\tdie(_(\"failed to clear multi-pack-index at %s\"), midx.buf);\n \n-\tclear_midx_files_ext(r->objects->sources->path, MIDX_EXT_BITMAP, NULL);\n-\tclear_midx_files_ext(r->objects->sources->path, MIDX_EXT_REV, NULL);\n+\tclear_midx_files_ext(r->objects->sources, MIDX_EXT_BITMAP, NULL);\n+\tclear_midx_files_ext(r->objects->sources, MIDX_EXT_REV, NULL);\n \n \tstrbuf_release(&midx);\n }\n@@ -888,7 +884,7 @@ int verify_midx_file(struct odb_source *source, unsigned flags)\n \t\tstruct stat sb;\n \t\tstruct strbuf filename = STRBUF_INIT;\n \n-\t\tget_midx_filename(r->hash_algo, &filename, source->path);\n+\t\tget_midx_filename(source, &filename);\n \n \t\tif (!stat(filename.buf, &sb)) {\n \t\t\terror(_(\"multi-pack-index file exists, but failed to parse\"));\ndiff --git a/midx.h b/midx.h\nindex 71dbdec66e..e241d2d690 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -86,15 +86,12 @@ struct multi_pack_index {\n #define MIDX_EXT_MIDX \"midx\"\n \n const unsigned char *get_midx_checksum(struct multi_pack_index *m);\n-void get_midx_filename(const struct git_hash_algo *hash_algo,\n-\t\t struct strbuf *out, const char *object_dir);\n-void get_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t struct strbuf *out, const char *object_dir,\n+void get_midx_filename(struct odb_source *source, struct strbuf *out);\n+void get_midx_filename_ext(struct odb_source *source, struct strbuf *out,\n \t\t\t const unsigned char *hash, const char *ext);\n-void get_midx_chain_dirname(struct strbuf *buf, const char *object_dir);\n-void get_midx_chain_filename(struct strbuf *buf, const char *object_dir);\n-void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t\t struct strbuf *buf, const char *object_dir,\n+void get_midx_chain_dirname(struct odb_source *source, struct strbuf *out);\n+void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);\n+void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,\n \t\t\t\t const unsigned char *hash, const char *ext);\n \n struct multi_pack_index *load_multi_pack_index(struct odb_source *source);\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 01e14c34bd..058bdb5d7d 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -418,13 +418,12 @@ char *midx_bitmap_filename(struct multi_pack_index *midx)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n \tif (midx->has_chain)\n-\t\tget_split_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t\t midx->source->path,\n+\t\tget_split_midx_filename_ext(midx->source, &buf,\n \t\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t\t MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t midx->source->path, get_midx_checksum(midx),\n+\t\tget_midx_filename_ext(midx->source, &buf,\n+\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t MIDX_EXT_BITMAP);\n \n \treturn strbuf_detach(&buf, NULL);\n@@ -463,8 +462,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tget_midx_filename(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t midx->source->path);\n+\t\tget_midx_filename(midx->source, &buf);\n \t\ttrace2_data_string(\"bitmap\", bitmap_repo(bitmap_git),\n \t\t\t\t \"ignoring extra midx bitmap file\", buf.buf);\n \t\tclose(fd);\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex b206518dcb..d0791cc493 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -389,12 +389,12 @@ int load_midx_revindex(struct multi_pack_index *m)\n \t\t\t \"source\", \"rev\");\n \n \tif (m->has_chain)\n-\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n-\t\t\t\t\t m->source->path, get_midx_checksum(m),\n+\t\tget_split_midx_filename_ext(m->source, &revindex_name,\n+\t\t\t\t\t get_midx_checksum(m),\n \t\t\t\t\t MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n-\t\t\t\t m->source->path, get_midx_checksum(m),\n+\t\tget_midx_filename_ext(m->source, &revindex_name,\n+\t\t\t\t get_midx_checksum(m),\n \t\t\t\t MIDX_EXT_REV);\n \n \tret = load_revindex_from_disk(m->source->odb->repo->hash_algo,\n\n-- \n2.51.0.rc0.215.g125493bb4a.dirty\n\n"},{"id":"523735","messageId":"CAOLa=ZTr6pkYHu05xBKJwY+6RL215o06c=qgZ1b=4MTMxyV6cg@mail.gmail.com","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-1-748db2eda3b5@pks.im","subject":"Re: [PATCH 1/8] odb: store locality in object database sources","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-08-07T08:12:37Z","receivedAt":"2025-08-07T08:12:39Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> Object database sources are classified either as:\n>\n> - Local, which means that the source is the repository's primary\n> source. This is typically \".git/objects\".\n>\n> - Non-local, which is everything else. Most importantly this includes\n> alternates and quarantine directories.\n>\n> This locality is often computed ad-hoc by checking whether a given\n> object source is the first one. This works, but it is quite roundabout.\n>\n> Refactor the code so that we store locality when creating the sources in\n> the first place. This makes it both more accessible and robust.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> midx.c | 5 +++--\n> midx.h | 2 +-\n> odb.c | 1 +\n> odb.h | 8 ++++++++\n> packfile.c | 9 ++++-----\n> repository.c | 1 +\n> 6 files changed, 18 insertions(+), 8 deletions(-)\n>\n> diff --git a/midx.c b/midx.c\n> index 7d407682e6..b9ca0915a6 100644\n> --- a/midx.c\n> +++ b/midx.c\n> @@ -723,7 +723,7 @@ int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id)\n> \treturn 0;\n> }\n>\n> -int prepare_multi_pack_index_one(struct odb_source *source, int local)\n> +int prepare_multi_pack_index_one(struct odb_source *source)\n> {\n> \tstruct repository *r = source->odb->repo;\n>\n> @@ -734,7 +734,8 @@ int prepare_multi_pack_index_one(struct odb_source *source, int local)\n> \tif (source->midx)\n> \t\treturn 1;\n>\n> -\tsource->midx = load_multi_pack_index(r, source->path, local);\n> +\tsource->midx = load_multi_pack_index(r, source->path,\n> +\t\t\t\t\t source->local);\n>\n> \treturn !!source->midx;\n> }\n> diff --git a/midx.h b/midx.h\n> index 076382de8a..28c426a823 100644\n> --- a/midx.h\n> +++ b/midx.h\n> @@ -122,7 +122,7 @@ int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pa\n> int midx_contains_pack(struct multi_pack_index *m,\n> \t\t const char *idx_or_pack_name);\n> int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);\n> -int prepare_multi_pack_index_one(struct odb_source *source, int local);\n> +int prepare_multi_pack_index_one(struct odb_source *source);\n>\n> /*\n> * Variant of write_midx_file which writes a MIDX containing only the packs\n> diff --git a/odb.c b/odb.c\n> index 1f48a0448e..1761a50840 100644\n> --- a/odb.c\n> +++ b/odb.c\n> @@ -176,6 +176,7 @@ static int link_alt_odb_entry(struct object_database *odb,\n>\n> \tCALLOC_ARRAY(alternate, 1);\n> \talternate->odb = odb;\n> +\talternate->local = false;\n> \t/* pathbuf.buf is already in r->objects->source_by_path */\n> \talternate->path = strbuf_detach(&pathbuf, NULL);\n>\n> diff --git a/odb.h b/odb.h\n> index 09177bf430..d9f4dcf79f 100644\n> --- a/odb.h\n> +++ b/odb.h\n> @@ -63,6 +63,14 @@ struct odb_source {\n> \t */\n> \tstruct multi_pack_index *midx;\n>\n> +\t/*\n> +\t * Figure out whether this is the local alternate of the owning\n> +\t * repository, which would typically be its \".git/objects\" directory.\n> +\t * This local object directory is usually where objects would be\n> +\t * written to.\n> +\t */\n> +\tbool local;\n> +\n\nSo now, each 'obd_source' has a 'local' field, which would be false by\ndefault. But when the local 'odb_source' is setup, we set it to true.\n\n> \t/*\n> \t * This is a temporary object store created by the tmp_objdir\n> \t * facility. Disable ref updates since the objects in the store\n> diff --git a/packfile.c b/packfile.c\n> index 5d73932f50..a38544b87b 100644\n> --- a/packfile.c\n> +++ b/packfile.c\n> @@ -935,14 +935,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,\n> \t\treport_garbage(PACKDIR_FILE_GARBAGE, full_name);\n> }\n>\n> -static void prepare_packed_git_one(struct odb_source *source, int local)\n> +static void prepare_packed_git_one(struct odb_source *source)\n> {\n> \tstruct string_list garbage = STRING_LIST_INIT_DUP;\n> \tstruct prepare_pack_data data = {\n> \t\t.m = source->midx,\n> \t\t.r = source->odb->repo,\n> \t\t.garbage = &garbage,\n> -\t\t.local = local,\n> +\t\t.local = source->local,\n>\n\nThe 'prepare_pack_data' already has a 'local' field, so we can now\nsimply set it using the 'odb_source's field of the same name. Nice.\n\n> \t};\n>\n> \tfor_each_file_in_pack_dir(source->path, prepare_pack, &data);\n> @@ -1037,9 +1037,8 @@ static void prepare_packed_git(struct repository *r)\n>\n> \todb_prepare_alternates(r->objects);\n> \tfor (source = r->objects->sources; source; source = source->next) {\n> -\t\tint local = (source == r->objects->sources);\n> -\t\tprepare_multi_pack_index_one(source, local);\n> -\t\tprepare_packed_git_one(source, local);\n> +\t\tprepare_multi_pack_index_one(source);\n> +\t\tprepare_packed_git_one(source);\n> \t}\n> \trearrange_packed_git(r);\n>\n> diff --git a/repository.c b/repository.c\n> index ecd691181f..97f0578381 100644\n> --- a/repository.c\n> +++ b/repository.c\n> @@ -168,6 +168,7 @@ void repo_set_gitdir(struct repository *repo,\n> \tif (!repo->objects->sources) {\n> \t\tCALLOC_ARRAY(repo->objects->sources, 1);\n> \t\trepo->objects->sources->odb = repo->objects;\n> +\t\trepo->objects->sources->local = true;\n> \t\trepo->objects->sources_tail = &repo->objects->sources->next;\n> \t}\n> \texpand_base_dir(&repo->objects->sources->path, o->object_dir,\n>\n\nSo this sets up the local 'odb_source' and as such sets 'local' to true.\n\nLooks good.\n\n> --\n> 2.50.1.619.g074bbf1d35.dirty\n"},{"id":"523736","messageId":"CAOLa=ZThnuczO3xXdVupaqmbQK6t8z30pE7krG2djS4Lfr=jyg@mail.gmail.com","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-3-748db2eda3b5@pks.im","subject":"Re: [PATCH 3/8] odb: return newly created in-memory sources","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-08-07T08:21:17Z","receivedAt":"2025-08-07T08:21:19Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> Callers have no trivial way to obtain the newly created object database\n> source when adding it to the in-memory list of alternates. While not yet\n> needed anywhere, a subsequent commit will want to obtain that pointer.\n>\n> Refactor the function to return the source to make it easily accessible.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> odb.c | 37 +++++++++++++++++++++----------------\n> odb.h | 4 ++--\n> 2 files changed, 23 insertions(+), 18 deletions(-)\n>\n> diff --git a/odb.c b/odb.c\n> index 61104b7cb8..7793816f81 100644\n> --- a/odb.c\n> +++ b/odb.c\n> @@ -139,23 +139,22 @@ static void read_info_alternates(struct object_database *odb,\n> \t\t\t\t const char *relative_base,\n> \t\t\t\t int depth);\n>\n> -static int link_alt_odb_entry(struct object_database *odb,\n> -\t\t\t const struct strbuf *entry,\n> -\t\t\t const char *relative_base,\n> -\t\t\t int depth,\n> -\t\t\t const char *normalized_objdir)\n> +static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n> +\t\t\t\t\t const char *entry,\n> +\t\t\t\t\t const char *relative_base,\n> +\t\t\t\t\t int depth,\n> +\t\t\t\t\t const char *normalized_objdir)\n\nThe change in return value makes sense, since we now want to return the\n'odb_source' newly created.\n\nBut the 'entry' argument is also changed here.\n\n> {\n> -\tstruct odb_source *alternate;\n> +\tstruct odb_source *alternate = NULL;\n> \tstruct strbuf pathbuf = STRBUF_INIT;\n> \tstruct strbuf tmp = STRBUF_INIT;\n> \tkhiter_t pos;\n> -\tint ret = -1;\n>\n> -\tif (!is_absolute_path(entry->buf) && relative_base) {\n> +\tif (!is_absolute_path(entry) && relative_base) {\n> \t\tstrbuf_realpath(&pathbuf, relative_base, 1);\n> \t\tstrbuf_addch(&pathbuf, '/');\n> \t}\n> -\tstrbuf_addbuf(&pathbuf, entry);\n> +\tstrbuf_addstr(&pathbuf, entry);\n>\n> \tif (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {\n> \t\terror(_(\"unable to normalize alternate object path: %s\"),\n> @@ -189,11 +188,11 @@ static int link_alt_odb_entry(struct object_database *odb,\n>\n> \t/* recursively add alternates */\n> \tread_info_alternates(odb, alternate->path, depth + 1);\n> -\tret = 0;\n> +\n> error:\n> \tstrbuf_release(&tmp);\n> \tstrbuf_release(&pathbuf);\n> -\treturn ret;\n> +\treturn alternate;\n> }\n>\n> static const char *parse_alt_odb_entry(const char *string,\n> @@ -246,7 +245,7 @@ static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n> \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n> \t\tif (!entry.len)\n> \t\t\tcontinue;\n> -\t\tlink_alt_odb_entry(odb, &entry,\n> +\t\tlink_alt_odb_entry(odb, entry.buf,\n> \t\t\t\t relative_base, depth, objdirbuf.buf);\n\nI'm assuming that passing 'const char *' directly makes the function\nmore versatile. Might be worthwhile to call this out in the commit\nmessage.\n\n[snip]\n"},{"id":"523737","messageId":"CAOLa=ZTA_3po038J7DGxk=gEM3JkkH5afBeRqTJLcKNM87hmxQ@mail.gmail.com","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-5-748db2eda3b5@pks.im","subject":"Re: [PATCH 5/8] midx: load multi-pack indices via their source","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-08-07T08:49:17Z","receivedAt":"2025-08-07T08:49:19Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> To load a multi-pack index the caller is expected to pass both the\n> repository and the object directory where the multi-pack index is\n> located. While this works, this layout has a couple of downsides:\n>\n> - We need to pass in information reduntant with the owning source,\n> namely its object directory and whether the source is local or not.\n>\n> - We don't have access to the source when loading the multi-pack\n> index. If we had that access, we could store a pointer to the owning\n> source in the MIDX and thus deduplicate some information.\n>\n> - Multi-pack indices are inherently specific to the object source and\n> its format. With the goal of pluggable object backends in mind we\n> will eventually want the backends to own the logic of reading and\n> writing multi-pack indices. Making the logic work on top of object\n> sources is a step into that direction.\n>\n> Refactor loading of multi-pack indices accordingly.\n>\n> This surfaces one small problem though: git-multi-pack-index(1) and our\n> MIDX test helper both know to read and write multi-pack-indices located\n> in a different object directory. This issue is addressed by adding the\n> user-provided object directory as an in-memory alternate.\n>\n\nDoesn't this commit only fix the 'read' side of things i.e.\n'cmd_multi_pack_index_verify'.\n\nShouldn't we squash the next commit into this? Otherwise, writing midx\npresent in a different object directory is broken as of this commit no?\nFor e.g. in 'cmd_multi_pack_index_expire' we call\n'handle_object_dir_option' which would add it as an alternate obd, but\nwe don't use the return value at all.\n\n[snip]\n"},{"id":"523738","messageId":"CAOLa=ZQxu6tCMyUWzmDnEnKs=ydsC3Av=UX6v_ZYyxZQBV3aEA@mail.gmail.com","threadId":"63871","inReplyTo":"CAOLa=ZTA_3po038J7DGxk=gEM3JkkH5afBeRqTJLcKNM87hmxQ@mail.gmail.com","subject":"Re: [PATCH 5/8] midx: load multi-pack indices via their source","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-08-07T08:51:33Z","receivedAt":"2025-08-07T08:51:35Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Karthik Nayak writes:\n\n> Patrick Steinhardt writes:\n>\n>> To load a multi-pack index the caller is expected to pass both the\n>> repository and the object directory where the multi-pack index is\n>> located. While this works, this layout has a couple of downsides:\n>>\n>> - We need to pass in information reduntant with the owning source,\n>> namely its object directory and whether the source is local or not.\n>>\n>> - We don't have access to the source when loading the multi-pack\n>> index. If we had that access, we could store a pointer to the owning\n>> source in the MIDX and thus deduplicate some information.\n>>\n>> - Multi-pack indices are inherently specific to the object source and\n>> its format. With the goal of pluggable object backends in mind we\n>> will eventually want the backends to own the logic of reading and\n>> writing multi-pack indices. Making the logic work on top of object\n>> sources is a step into that direction.\n>>\n>> Refactor loading of multi-pack indices accordingly.\n>>\n>> This surfaces one small problem though: git-multi-pack-index(1) and our\n>> MIDX test helper both know to read and write multi-pack-indices located\n>> in a different object directory. This issue is addressed by adding the\n>> user-provided object directory as an in-memory alternate.\n>>\n>\n> Doesn't this commit only fix the 'read' side of things i.e.\n> 'cmd_multi_pack_index_verify'.\n>\n> Shouldn't we squash the next commit into this? Otherwise, writing midx\n> present in a different object directory is broken as of this commit no?\n> For e.g. in 'cmd_multi_pack_index_expire' we call\n> 'handle_object_dir_option' which would add it as an alternate obd, but\n> we don't use the return value at all.\n>\n> [snip]\n\nReading into the next commit, I see these paths internally call\n'lookup_multi_pack_index' which would find the right source, so this\ncommit does work as expected. So all good.\n"},{"id":"523739","messageId":"CAOLa=ZTdgra4j-9WbU8nnyQ=38W03MRWbYdJgCGpxnSkumJ9Kg@mail.gmail.com","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-6-748db2eda3b5@pks.im","subject":"Re: [PATCH 6/8] midx: write multi-pack indices via their source","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-08-07T08:55:20Z","receivedAt":"2025-08-07T08:55:22Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n[snip]\n\n> diff --git a/midx-write.c b/midx-write.c\n> index 37a0b1682f..bf7c01d4b1 100644\n> --- a/midx-write.c\n> +++ b/midx-write.c\n> @@ -913,15 +913,6 @@ static int write_midx_bitmap(struct write_midx_context *ctx,\n> \treturn ret;\n> }\n>\n> -static struct multi_pack_index *lookup_multi_pack_index(struct repository *r,\n> -\t\t\t\t\t\t\tconst char *object_dir)\n> -{\n> -\tstruct odb_source *source = odb_find_source(r->objects, object_dir);\n> -\tif (!source)\n> -\t\tdie(_(\"could not find object directory matching %s\"), object_dir);\n> -\treturn get_multi_pack_index(source);\n> -}\n> -\n>\n\nSince we now directly pass the source, there is no need to find it from\nthe repository. Makes sense.\n\n[snip]\n\nThe rest of the patch, simply replaces the 'repository' object with the\n'odb_source' object. This makes things a lot more contained and is\nindeed a much nicer flow of information.\n"},{"id":"523740","messageId":"CAOLa=ZRvA2kLd-EfymOTF=SvyTFR+Xw1QSvDH4BUW2rbz4FmAw@mail.gmail.com","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"Re: [PATCH 0/8] midx: stop deduplicating info redundant with their sources","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2025-08-07T08:58:32Z","receivedAt":"2025-08-07T08:58:33Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"Patrick Steinhardt writes:\n\n> Hi,\n>\n> this patch series is a follow up to the recent patch series that started\n> to move the MIDX into its object source. It refactors the MIDX subsystem\n> so that we always read and write them via an ODB source. This ultimately\n> allows us to store a pointer to the owning ODB source in the MIDX and\n> thus dedup some redundant information.\n>\n> This series is built on top of e813a0200a7 (The fifteenth batch,\n> 2025-07-28) with ps/object-store-midx at ec865d94d46 (midx: remove\n> now-unused linked list of multi-pack indices, 2025-07-15) merged into\n> it.\n>\n> Thanks!\n>\n> Patrick\n>\n\nI went through the patches and while I don't have a lot of context in\nthis section of the codebase, the changes seemed good to me! Thanks\n\nKarthik\n"},{"id":"523779","messageId":"aJUkX0SRmBWLS6rQ@nand.local","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-1-bcffb8fc119c@pks.im","subject":"Re: [PATCH v2 1/9] odb: store locality in object database sources","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-07T22:10:39Z","receivedAt":"2025-08-07T22:10:42Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 07, 2025 at 10:09:51AM +0200, Patrick Steinhardt wrote:\n> Object database sources are classified either as:\n>\n> - Local, which means that the source is the repository's primary\n> source. This is typically \".git/objects\".\n>\n> - Non-local, which is everything else. Most importantly this includes\n> alternates and quarantine directories.\n>\n> This locality is often computed ad-hoc by checking whether a given\n> object source is the first one. This works, but it is quite roundabout.\n>\n> Refactor the code so that we store locality when creating the sources in\n> the first place. This makes it both more accessible and robust.\n\nLooking good. The only caller here is packfile.c::prepare_packed_git(),\nwhich passes local when \"source == r->objects->sources\". Assuming that\n->local is faithfully set to true in that case, this change looks good\nto me.\n\nThanks,\nTaylor\n"},{"id":"523780","messageId":"aJUk68fgQttR2gMe@nand.local","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-2-bcffb8fc119c@pks.im","subject":"Re: [PATCH v2 2/9] odb: allow `odb_find_source()` to fail","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-07T22:12:59Z","receivedAt":"2025-08-07T22:13:01Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 07, 2025 at 10:09:52AM +0200, Patrick Steinhardt wrote:\n> When trying to locate a source for an unknown object directory we will\n> die right away. In subsequent patches we will add new callsites though\n> that want to handle this situation gracefully instead.\n>\n> Refactor the function to return a `NULL` pointer if the source could not\n> be found and adapt the callsites to die instead.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> builtin/commit-graph.c | 4 ++++\n> midx-write.c | 2 ++\n> odb.c | 2 --\n> odb.h | 4 ++--\n> 4 files changed, 8 insertions(+), 4 deletions(-)\n>\n> diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\n> index 25018a0b9d..dc2c1a5432 100644\n> --- a/builtin/commit-graph.c\n> +++ b/builtin/commit-graph.c\n> @@ -102,6 +102,8 @@ static int graph_verify(int argc, const char **argv, const char *prefix,\n> \t\tflags |= COMMIT_GRAPH_WRITE_PROGRESS;\n>\n> \tsource = odb_find_source(the_repository->objects, opts.obj_dir);\n> +\tif (!source)\n> +\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n\nMakes sense, and I am glad that we are trending towards having fewer\ninternal functions that assume the caller wants to die() on failure. I\nwonder if it might be worth having a odb_find_source_or_die()\ncounterpart here such that callers don't have to repeat the \"could not\nfind object directory ...\" message.\n\nPerhaps something like:\n\n struct odb_source *odb_find_source_or_die(struct object_database *odb,\n const char *obj_dir)\n {\n struct odb_source *source = odb_find_source(odb, obj_dir);\n if (!source)\n die(_(\"could not find object directory matching %s\"),\n obj_dir);\n return source;\n }\n\n?\n\nOf course, callers that want to use a different message or otherwise\nhandle a missing source differently would still be able to do so by\ncalling odb_find_source() directly.\n\nThanks,\nTaylor\n"},{"id":"523781","messageId":"aJUlwLJyxSmD4EQF@nand.local","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-3-bcffb8fc119c@pks.im","subject":"Re: [PATCH v2 3/9] odb: return newly created in-memory sources","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-07T22:16:32Z","receivedAt":"2025-08-07T22:16:35Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 07, 2025 at 10:09:53AM +0200, Patrick Steinhardt wrote:\n> Callers have no trivial way to obtain the newly created object database\n> source when adding it to the in-memory list of alternates. While not yet\n> needed anywhere, a subsequent commit will want to obtain that pointer.\n>\n> Refactor the function to return the source to make it easily accessible.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> odb.c | 37 +++++++++++++++++++++----------------\n> odb.h | 4 ++--\n> 2 files changed, 23 insertions(+), 18 deletions(-)\n>\n> diff --git a/odb.c b/odb.c\n> index 61104b7cb8..7793816f81 100644\n> --- a/odb.c\n> +++ b/odb.c\n> @@ -139,23 +139,22 @@ static void read_info_alternates(struct object_database *odb,\n> \t\t\t\t const char *relative_base,\n> \t\t\t\t int depth);\n>\n> -static int link_alt_odb_entry(struct object_database *odb,\n> -\t\t\t const struct strbuf *entry,\n> -\t\t\t const char *relative_base,\n> -\t\t\t int depth,\n> -\t\t\t const char *normalized_objdir)\n> +static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n> +\t\t\t\t\t const char *entry,\n> +\t\t\t\t\t const char *relative_base,\n> +\t\t\t\t\t int depth,\n> +\t\t\t\t\t const char *normalized_objdir)\n\nBeyond returning an odb_source pointer from this function, it looks like\n\"entry\" was changed form a strbuf to a bare const char *. Is that\nintentional? If so, it might be worth mentioning it in the commit\nmessage, or optionally doing that as a preparatory step.\n\n> @@ -316,17 +315,23 @@ void odb_add_to_alternates_file(struct object_database *odb,\n> \tfree(alts);\n> }\n>\n> -void odb_add_to_alternates_memory(struct object_database *odb,\n> -\t\t\t\t const char *reference)\n> +struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n> +\t\t\t\t\t\tconst char *reference)\n\nNot the fault of your patch, but while we're refactoring, do you think\nit makes sense to rename the second parameter here to \"dir\", or changing\n\"dir\" to \"reference\" in the declaration of this function over in odb.h?\n\nThanks,\nTaylor\n"},{"id":"523782","messageId":"aJUm9vU9uc7MaP6X@nand.local","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-4-bcffb8fc119c@pks.im","subject":"Re: [PATCH v2 4/9] odb: simplify calling `link_alt_odb_entry()`","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-07T22:21:42Z","receivedAt":"2025-08-07T22:21:45Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 07, 2025 at 10:09:54AM +0200, Patrick Steinhardt wrote:\n> ---\n> odb.c | 25 +++++++------------------\n> 1 file changed, 7 insertions(+), 18 deletions(-)\n\nVery nice, this was a quite pleasant read.\n\nThanks,\nTaylor\n"},{"id":"523783","messageId":"aJUn0qeliNQ/nnWr@nand.local","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-6-bcffb8fc119c@pks.im","subject":"Re: [PATCH v2 6/9] midx: load multi-pack indices via their source","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-07T22:25:22Z","receivedAt":"2025-08-07T22:25:24Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 07, 2025 at 10:09:56AM +0200, Patrick Steinhardt wrote:\n> To load a multi-pack index the caller is expected to pass both the\n> repository and the object directory where the multi-pack index is\n> located. While this works, this layout has a couple of downsides:\n>\n> - We need to pass in information reduntant with the owning source,\n> namely its object directory and whether the source is local or not.\n>\n> - We don't have access to the source when loading the multi-pack\n> index. If we had that access, we could store a pointer to the owning\n> source in the MIDX and thus deduplicate some information.\n>\n> - Multi-pack indices are inherently specific to the object source and\n> its format. With the goal of pluggable object backends in mind we\n> will eventually want the backends to own the logic of reading and\n> writing multi-pack indices. Making the logic work on top of object\n> sources is a step into that direction.\n>\n> Refactor loading of multi-pack indices accordingly.\n>\n> This surfaces one small problem though: git-multi-pack-index(1) and our\n> MIDX test helper both know to read and write multi-pack-indices located\n> in a different object directory. This issue is addressed by adding the\n> user-provided object directory as an in-memory alternate.\n>\n> Signed-off-by: Patrick Steinhardt \n> ---\n> builtin/multi-pack-index.c | 18 ++++++++++++--\n> midx.c | 57 ++++++++++++++++++++-------------------------\n> midx.h | 6 ++---\n> t/helper/test-read-midx.c | 25 ++++++++++++--------\n> t/t5319-multi-pack-index.sh | 8 +++----\n> 5 files changed, 62 insertions(+), 52 deletions(-)\n>\n> diff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c\n> index aa25b06f9d..e4a9305af3 100644\n> --- a/builtin/multi-pack-index.c\n> +++ b/builtin/multi-pack-index.c\n> @@ -64,12 +64,20 @@ static int parse_object_dir(const struct option *opt, const char *arg,\n> \tchar **value = opt->value;\n> \tfree(*value);\n> \tif (unset)\n> -\t\t*value = xstrdup(repo_get_object_directory(the_repository));\n> +\t\t*value = xstrdup(the_repository->objects->sources->path);\n> \telse\n> \t\t*value = real_pathdup(arg, 1);\n> \treturn 0;\n> }\n>\n> +static struct odb_source *handle_object_dir_option(struct repository *repo)\n> +{\n> +\tstruct odb_source *source = odb_find_source(repo->objects, opts.object_dir);\n> +\tif (!source)\n> +\t\tsource = odb_add_to_alternates_memory(repo->objects, opts.object_dir);\n> +\treturn source;\n> +}\n> +\n\nHmm... I admit that I am pretty unfamiliar with the --object-dir option\nhere and only have a vague recollection of why it was added in the first\nplace.\n\nI am not sure one way or another if adding the user-provided directory\nas an in-memory alternate is the right thing to do here.\n\nStolee (CC'd) would you mind sharing your thoughts on this?\n\nThanks,\nTaylor\n"},{"id":"523784","messageId":"aJUn4Ggk6iSIfw1x@nand.local","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-7-bcffb8fc119c@pks.im","subject":"Re: [PATCH v2 7/9] midx: write multi-pack indices via their source","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-07T22:25:36Z","receivedAt":"2025-08-07T22:25:38Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 07, 2025 at 10:09:57AM +0200, Patrick Steinhardt wrote:\n> ---\n> builtin/multi-pack-index.c | 19 +++++++------\n> builtin/repack.c | 2 +-\n> midx-write.c | 69 +++++++++++++++++++++-------------------------\n> midx.h | 8 +++---\n> 4 files changed, 47 insertions(+), 51 deletions(-)\n\nLooks reasonable.\n\nThanks,\nTaylor\n"},{"id":"523785","messageId":"aJUoPJrVfGS5l9L1@nand.local","threadId":"63871","inReplyTo":"20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im","subject":"Re: [PATCH v2 0/9] midx: stop duplicating info redundant with their sources","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-07T22:27:08Z","receivedAt":"2025-08-07T22:27:10Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 07, 2025 at 10:09:50AM +0200, Patrick Steinhardt wrote:\n> ---\n> Patrick Steinhardt (9):\n> odb: store locality in object database sources\n> odb: allow `odb_find_source()` to fail\n> odb: return newly created in-memory sources\n> odb: simplify calling `link_alt_odb_entry()`\n> midx: drop redundant `struct repository` parameter\n> midx: load multi-pack indices via their source\n> midx: write multi-pack indices via their source\n> midx: stop duplicating info redundant with its owning source\n> midx: compute paths via their source\n\nI read through these patches, and they look pretty good to me. I left a\nfew minor comments in the first half of the series, but nothing\nshow-stopping there.\n\nI would, however, like to hear from Stolee on the --object-dir stuff,\nsince I am not sure if the implementation here has any unintended\nconsequences. That feature is (as yet) mysterious to me.\n\nThanks,\nTaylor\n"},{"id":"523924","messageId":"aJnabZWb9WBQK2KF@pks.im","threadId":"63871","inReplyTo":"aJUk68fgQttR2gMe@nand.local","subject":"Re: [PATCH v2 2/9] odb: allow `odb_find_source()` to fail","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T11:56:29Z","receivedAt":"2025-08-11T11:56:37Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Aug 07, 2025 at 06:12:59PM -0400, Taylor Blau wrote:\n> On Thu, Aug 07, 2025 at 10:09:52AM +0200, Patrick Steinhardt wrote:\n> > When trying to locate a source for an unknown object directory we will\n> > die right away. In subsequent patches we will add new callsites though\n> > that want to handle this situation gracefully instead.\n> >\n> > Refactor the function to return a `NULL` pointer if the source could not\n> > be found and adapt the callsites to die instead.\n> >\n> > Signed-off-by: Patrick Steinhardt \n> > ---\n> > builtin/commit-graph.c | 4 ++++\n> > midx-write.c | 2 ++\n> > odb.c | 2 --\n> > odb.h | 4 ++--\n> > 4 files changed, 8 insertions(+), 4 deletions(-)\n> >\n> > diff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\n> > index 25018a0b9d..dc2c1a5432 100644\n> > --- a/builtin/commit-graph.c\n> > +++ b/builtin/commit-graph.c\n> > @@ -102,6 +102,8 @@ static int graph_verify(int argc, const char **argv, const char *prefix,\n> > \t\tflags |= COMMIT_GRAPH_WRITE_PROGRESS;\n> >\n> > \tsource = odb_find_source(the_repository->objects, opts.obj_dir);\n> > +\tif (!source)\n> > +\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n> \n> Makes sense, and I am glad that we are trending towards having fewer\n> internal functions that assume the caller wants to die() on failure. I\n> wonder if it might be worth having a odb_find_source_or_die()\n> counterpart here such that callers don't have to repeat the \"could not\n> find object directory ...\" message.\n> \n> Perhaps something like:\n> \n> struct odb_source *odb_find_source_or_die(struct object_database *odb,\n> const char *obj_dir)\n> {\n> struct odb_source *source = odb_find_source(odb, obj_dir);\n> if (!source)\n> die(_(\"could not find object directory matching %s\"),\n> obj_dir);\n> return source;\n> }\n> \n> ?\n> \n> Of course, callers that want to use a different message or otherwise\n> handle a missing source differently would still be able to do so by\n> calling odb_find_source() directly.\n\nFair, that'd allow us to get rid of the repeated calls to `die()`. Will\ndo, thanks!\n\nPatrick\n"},{"id":"523925","messageId":"aJnaddzo3aRbmEta@pks.im","threadId":"63871","inReplyTo":"aJUlwLJyxSmD4EQF@nand.local","subject":"Re: [PATCH v2 3/9] odb: return newly created in-memory sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T11:56:37Z","receivedAt":"2025-08-11T11:56:43Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Aug 07, 2025 at 06:16:32PM -0400, Taylor Blau wrote:\n> On Thu, Aug 07, 2025 at 10:09:53AM +0200, Patrick Steinhardt wrote:\n> > Callers have no trivial way to obtain the newly created object database\n> > source when adding it to the in-memory list of alternates. While not yet\n> > needed anywhere, a subsequent commit will want to obtain that pointer.\n> >\n> > Refactor the function to return the source to make it easily accessible.\n> >\n> > Signed-off-by: Patrick Steinhardt \n> > ---\n> > odb.c | 37 +++++++++++++++++++++----------------\n> > odb.h | 4 ++--\n> > 2 files changed, 23 insertions(+), 18 deletions(-)\n> >\n> > diff --git a/odb.c b/odb.c\n> > index 61104b7cb8..7793816f81 100644\n> > --- a/odb.c\n> > +++ b/odb.c\n> > @@ -139,23 +139,22 @@ static void read_info_alternates(struct object_database *odb,\n> > \t\t\t\t const char *relative_base,\n> > \t\t\t\t int depth);\n> >\n> > -static int link_alt_odb_entry(struct object_database *odb,\n> > -\t\t\t const struct strbuf *entry,\n> > -\t\t\t const char *relative_base,\n> > -\t\t\t int depth,\n> > -\t\t\t const char *normalized_objdir)\n> > +static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n> > +\t\t\t\t\t const char *entry,\n> > +\t\t\t\t\t const char *relative_base,\n> > +\t\t\t\t\t int depth,\n> > +\t\t\t\t\t const char *normalized_objdir)\n> \n> Beyond returning an odb_source pointer from this function, it looks like\n> \"entry\" was changed form a strbuf to a bare const char *. Is that\n> intentional? If so, it might be worth mentioning it in the commit\n> message, or optionally doing that as a preparatory step.\n\nDone now.\n\n> > @@ -316,17 +315,23 @@ void odb_add_to_alternates_file(struct object_database *odb,\n> > \tfree(alts);\n> > }\n> >\n> > -void odb_add_to_alternates_memory(struct object_database *odb,\n> > -\t\t\t\t const char *reference)\n> > +struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n> > +\t\t\t\t\t\tconst char *reference)\n> \n> Not the fault of your patch, but while we're refactoring, do you think\n> it makes sense to rename the second parameter here to \"dir\", or changing\n> \"dir\" to \"reference\" in the declaration of this function over in odb.h?\n\nAh, these are inconsistent indeed, good catch. Will fix.\n\nPatrick\n"},{"id":"523926","messageId":"aJnaf_7i08pGvV43@pks.im","threadId":"63871","inReplyTo":"aJUoPJrVfGS5l9L1@nand.local","subject":"Re: [PATCH v2 0/9] midx: stop duplicating info redundant with their sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T11:56:47Z","receivedAt":"2025-08-11T11:56:51Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Thu, Aug 07, 2025 at 06:27:08PM -0400, Taylor Blau wrote:\n> On Thu, Aug 07, 2025 at 10:09:50AM +0200, Patrick Steinhardt wrote:\n> > ---\n> > Patrick Steinhardt (9):\n> > odb: store locality in object database sources\n> > odb: allow `odb_find_source()` to fail\n> > odb: return newly created in-memory sources\n> > odb: simplify calling `link_alt_odb_entry()`\n> > midx: drop redundant `struct repository` parameter\n> > midx: load multi-pack indices via their source\n> > midx: write multi-pack indices via their source\n> > midx: stop duplicating info redundant with its owning source\n> > midx: compute paths via their source\n> \n> I read through these patches, and they look pretty good to me. I left a\n> few minor comments in the first half of the series, but nothing\n> show-stopping there.\n> \n> I would, however, like to hear from Stolee on the --object-dir stuff,\n> since I am not sure if the implementation here has any unintended\n> consequences. That feature is (as yet) mysterious to me.\n\nThanks for your review! I'll reroll even without having heard from\nStolee just yet, but will add him to Cc for the next iteration.\n\nPatrick\n"},{"id":"523934","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im","subject":"[PATCH v3 00/10] midx: stop duplicating info redundant with their sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:40Z","receivedAt":"2025-08-11T13:46:50Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series is a follow up to the recent patch series that started\nto move the MIDX into its object source. It refactors the MIDX subsystem\nso that we always read and write them via an ODB source. This ultimately\nallows us to store a pointer to the owning ODB source in the MIDX and\nthus dedup some redundant information.\n\nThis series is built on top of e813a0200a7 (The fifteenth batch,\n2025-07-28) with ps/object-store-midx at ec865d94d46 (midx: remove\nnow-unused linked list of multi-pack indices, 2025-07-15) merged into\nit.\n\nChanges in v2:\n - Fix a comment typo.\n - Introduce another commit that simplifies the calling convention for\n `link_alt_odb_entry()`.\n - Link to v1: https://lore.kernel.org/r/20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im\n\nChanges in v3:\n - Introduce `odb_find_source_or_die()` so that we don't have to repeat\n the calls to `die()`, as suggested by Taylor.\n - Split out a patch to adapt `link_alt_odb_entry()` and friends to\n consistently name the parameter that refers to the alternate object\n directory's path.\n - Link to v2: https://lore.kernel.org/r/20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (10):\n odb: store locality in object database sources\n odb: allow `odb_find_source()` to fail\n odb: consistently use \"dir\" to refer to alternate's directory\n odb: return newly created in-memory sources\n odb: simplify calling `link_alt_odb_entry()`\n midx: drop redundant `struct repository` parameter\n midx: load multi-pack indices via their source\n midx: write multi-pack indices via their source\n midx: stop duplicating info redundant with its owning source\n midx: compute paths via their source\n\n builtin/commit-graph.c | 4 +-\n builtin/multi-pack-index.c | 31 +++++++---\n builtin/pack-objects.c | 2 +-\n builtin/repack.c | 7 ++-\n midx-write.c | 116 +++++++++++++++++--------------------\n midx.c | 135 +++++++++++++++++++++-----------------------\n midx.h | 42 ++++++--------\n odb.c | 64 ++++++++++-----------\n odb.h | 19 +++++--\n pack-bitmap.c | 15 +++--\n pack-revindex.c | 14 ++---\n packfile.c | 13 ++---\n repository.c | 1 +\n t/helper/test-read-midx.c | 31 +++++-----\n t/t5319-multi-pack-index.sh | 8 +--\n 15 files changed, 254 insertions(+), 248 deletions(-)\n\nRange-diff versus v2:\n\n 1: 937fe12914 = 1: 51817fdd71 odb: store locality in object database sources\n 2: c1a382e2e5 < -: ---------- odb: allow `odb_find_source()` to fail\n -: ---------- > 2: 37da6f7908 odb: allow `odb_find_source()` to fail\n -: ---------- > 3: b0f3c8af7b odb: consistently use \"dir\" to refer to alternate's directory\n 3: 31fa81ce51 ! 4: 88af2d77bd odb: return newly created in-memory sources\n @@ odb.c: static void read_info_alternates(struct object_database *odb,\n \t\t\t\t int depth);\n \n -static int link_alt_odb_entry(struct object_database *odb,\n --\t\t\t const struct strbuf *entry,\n +-\t\t\t const char *dir,\n -\t\t\t const char *relative_base,\n -\t\t\t int depth,\n -\t\t\t const char *normalized_objdir)\n +static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n -+\t\t\t\t\t const char *entry,\n ++\t\t\t\t\t const char *dir,\n +\t\t\t\t\t const char *relative_base,\n +\t\t\t\t\t int depth,\n +\t\t\t\t\t const char *normalized_objdir)\n @@ odb.c: static void read_info_alternates(struct object_database *odb,\n \tkhiter_t pos;\n -\tint ret = -1;\n \n --\tif (!is_absolute_path(entry->buf) && relative_base) {\n -+\tif (!is_absolute_path(entry) && relative_base) {\n + \tif (!is_absolute_path(dir) && relative_base) {\n \t\tstrbuf_realpath(&pathbuf, relative_base, 1);\n - \t\tstrbuf_addch(&pathbuf, '/');\n - \t}\n --\tstrbuf_addbuf(&pathbuf, entry);\n -+\tstrbuf_addstr(&pathbuf, entry);\n - \n - \tif (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {\n - \t\terror(_(\"unable to normalize alternate object path: %s\"),\n @@ odb.c: static int link_alt_odb_entry(struct object_database *odb,\n \n \t/* recursively add alternates */\n @@ odb.c: static int link_alt_odb_entry(struct object_database *odb,\n }\n \n static const char *parse_alt_odb_entry(const char *string,\n -@@ odb.c: static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n - \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n - \t\tif (!entry.len)\n - \t\t\tcontinue;\n --\t\tlink_alt_odb_entry(odb, &entry,\n -+\t\tlink_alt_odb_entry(odb, entry.buf,\n - \t\t\t\t relative_base, depth, objdirbuf.buf);\n - \t}\n - \tstrbuf_release(&entry);\n @@ odb.c: void odb_add_to_alternates_file(struct object_database *odb,\n \tfree(alts);\n }\n \n -void odb_add_to_alternates_memory(struct object_database *odb,\n --\t\t\t\t const char *reference)\n +-\t\t\t\t const char *dir)\n +struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n -+\t\t\t\t\t\tconst char *reference)\n ++\t\t\t\t\t\tconst char *dir)\n {\n +\tstruct odb_source *alternate;\n +\tchar *objdir;\n @@ odb.c: void odb_add_to_alternates_file(struct object_database *odb,\n \t */\n \todb_prepare_alternates(odb);\n \n --\tlink_alt_odb_entries(odb, reference,\n --\t\t\t '\\n', NULL, 0);\n +-\tlink_alt_odb_entries(odb, dir, '\\n', NULL, 0);\n +\tobjdir = real_pathdup(odb->sources->path, 1);\n -+\talternate = link_alt_odb_entry(odb, reference, NULL, 0, objdir);\n ++\talternate = link_alt_odb_entry(odb, dir, NULL, 0, objdir);\n +\n +\tfree(objdir);\n +\treturn alternate;\n 4: a94136e0a6 ! 5: 4f002abcee odb: simplify calling `link_alt_odb_entry()`\n @@ Commit message\n ## odb.c ##\n @@ odb.c: static void read_info_alternates(struct object_database *odb,\n static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n - \t\t\t\t\t const char *entry,\n + \t\t\t\t\t const char *dir,\n \t\t\t\t\t const char *relative_base,\n -\t\t\t\t\t int depth,\n -\t\t\t\t\t const char *normalized_objdir)\n @@ odb.c: static const char *parse_alt_odb_entry(const char *string,\n \t\t\t\t int sep, const char *relative_base, int depth)\n {\n -\tstruct strbuf objdirbuf = STRBUF_INIT;\n - \tstruct strbuf entry = STRBUF_INIT;\n + \tstruct strbuf dir = STRBUF_INIT;\n \n \tif (!alt || !*alt)\n @@ odb.c: static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n @@ odb.c: static void link_alt_odb_entries(struct object_database *odb, const char\n -\tstrbuf_realpath(&objdirbuf, odb->sources->path, 1);\n -\n \twhile (*alt) {\n - \t\talt = parse_alt_odb_entry(alt, sep, &entry);\n - \t\tif (!entry.len)\n + \t\talt = parse_alt_odb_entry(alt, sep, &dir);\n + \t\tif (!dir.len)\n \t\t\tcontinue;\n --\t\tlink_alt_odb_entry(odb, entry.buf,\n +-\t\tlink_alt_odb_entry(odb, dir.buf,\n -\t\t\t\t relative_base, depth, objdirbuf.buf);\n -+\t\tlink_alt_odb_entry(odb, entry.buf, relative_base, depth);\n ++\t\tlink_alt_odb_entry(odb, dir.buf, relative_base, depth);\n \t}\n - \tstrbuf_release(&entry);\n + \tstrbuf_release(&dir);\n -\tstrbuf_release(&objdirbuf);\n }\n \n static void read_info_alternates(struct object_database *odb,\n @@ odb.c: void odb_add_to_alternates_file(struct object_database *odb,\n struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n - \t\t\t\t\t\tconst char *reference)\n + \t\t\t\t\t\tconst char *dir)\n {\n -\tstruct odb_source *alternate;\n -\tchar *objdir;\n @@ odb.c: void odb_add_to_alternates_file(struct object_database *odb,\n \todb_prepare_alternates(odb);\n -\n -\tobjdir = real_pathdup(odb->sources->path, 1);\n --\talternate = link_alt_odb_entry(odb, reference, NULL, 0, objdir);\n +-\talternate = link_alt_odb_entry(odb, dir, NULL, 0, objdir);\n -\n -\tfree(objdir);\n -\treturn alternate;\n -+\treturn link_alt_odb_entry(odb, reference, NULL, 0);\n ++\treturn link_alt_odb_entry(odb, dir, NULL, 0);\n }\n \n struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,\n 5: eaf7cc324a = 6: 91cf3fc19a midx: drop redundant `struct repository` parameter\n 6: 64d1e33a88 = 7: eac8ce2f0d midx: load multi-pack indices via their source\n 7: 094fb9fd83 ! 8: 634ccedddf midx: write multi-pack indices via their source\n @@ midx-write.c: static int write_midx_bitmap(struct write_midx_context *ctx,\n -static struct multi_pack_index *lookup_multi_pack_index(struct repository *r,\n -\t\t\t\t\t\t\tconst char *object_dir)\n -{\n --\tstruct odb_source *source = odb_find_source(r->objects, object_dir);\n --\tif (!source)\n --\t\tdie(_(\"could not find object directory matching %s\"), object_dir);\n +-\tstruct odb_source *source = odb_find_source_or_die(r->objects, object_dir);\n -\treturn get_multi_pack_index(source);\n -}\n -\n 8: 2d6c844618 = 9: 91a15ad630 midx: stop duplicating info redundant with its owning source\n 9: 93f60b02b6 = 10: 357f6a4986 midx: compute paths via their source\n\n---\nbase-commit: ee2fd3d4b224c8145eeff04101ea6075560973e1\nchange-id: 20250729-b4-pks-midx-deduplicate-source-info-fcf3c08bfd53\n\n"},{"id":"523935","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-1-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 01/10] odb: store locality in object database sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:41Z","receivedAt":"2025-08-11T13:46:52Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Object database sources are classified either as:\n\n - Local, which means that the source is the repository's primary\n source. This is typically \".git/objects\".\n\n - Non-local, which is everything else. Most importantly this includes\n alternates and quarantine directories.\n\nThis locality is often computed ad-hoc by checking whether a given\nobject source is the first one. This works, but it is quite roundabout.\n\nRefactor the code so that we store locality when creating the sources in\nthe first place. This makes it both more accessible and robust.\n\nSigned-off-by: Patrick Steinhardt \n---\n midx.c | 5 +++--\n midx.h | 2 +-\n odb.c | 1 +\n odb.h | 8 ++++++++\n packfile.c | 9 ++++-----\n repository.c | 1 +\n 6 files changed, 18 insertions(+), 8 deletions(-)\n\ndiff --git a/midx.c b/midx.c\nindex 7d407682e6..b9ca0915a6 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -723,7 +723,7 @@ int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id)\n \treturn 0;\n }\n \n-int prepare_multi_pack_index_one(struct odb_source *source, int local)\n+int prepare_multi_pack_index_one(struct odb_source *source)\n {\n \tstruct repository *r = source->odb->repo;\n \n@@ -734,7 +734,8 @@ int prepare_multi_pack_index_one(struct odb_source *source, int local)\n \tif (source->midx)\n \t\treturn 1;\n \n-\tsource->midx = load_multi_pack_index(r, source->path, local);\n+\tsource->midx = load_multi_pack_index(r, source->path,\n+\t\t\t\t\t source->local);\n \n \treturn !!source->midx;\n }\ndiff --git a/midx.h b/midx.h\nindex 076382de8a..28c426a823 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -122,7 +122,7 @@ int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pa\n int midx_contains_pack(struct multi_pack_index *m,\n \t\t const char *idx_or_pack_name);\n int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);\n-int prepare_multi_pack_index_one(struct odb_source *source, int local);\n+int prepare_multi_pack_index_one(struct odb_source *source);\n \n /*\n * Variant of write_midx_file which writes a MIDX containing only the packs\ndiff --git a/odb.c b/odb.c\nindex 1f48a0448e..1761a50840 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -176,6 +176,7 @@ static int link_alt_odb_entry(struct object_database *odb,\n \n \tCALLOC_ARRAY(alternate, 1);\n \talternate->odb = odb;\n+\talternate->local = false;\n \t/* pathbuf.buf is already in r->objects->source_by_path */\n \talternate->path = strbuf_detach(&pathbuf, NULL);\n \ndiff --git a/odb.h b/odb.h\nindex 09177bf430..f9300439ba 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -63,6 +63,14 @@ struct odb_source {\n \t */\n \tstruct multi_pack_index *midx;\n \n+\t/*\n+\t * Figure out whether this is the local source of the owning\n+\t * repository, which would typically be its \".git/objects\" directory.\n+\t * This local object directory is usually where objects would be\n+\t * written to.\n+\t */\n+\tbool local;\n+\n \t/*\n \t * This is a temporary object store created by the tmp_objdir\n \t * facility. Disable ref updates since the objects in the store\ndiff --git a/packfile.c b/packfile.c\nindex 5d73932f50..a38544b87b 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -935,14 +935,14 @@ static void prepare_pack(const char *full_name, size_t full_name_len,\n \t\treport_garbage(PACKDIR_FILE_GARBAGE, full_name);\n }\n \n-static void prepare_packed_git_one(struct odb_source *source, int local)\n+static void prepare_packed_git_one(struct odb_source *source)\n {\n \tstruct string_list garbage = STRING_LIST_INIT_DUP;\n \tstruct prepare_pack_data data = {\n \t\t.m = source->midx,\n \t\t.r = source->odb->repo,\n \t\t.garbage = &garbage,\n-\t\t.local = local,\n+\t\t.local = source->local,\n \t};\n \n \tfor_each_file_in_pack_dir(source->path, prepare_pack, &data);\n@@ -1037,9 +1037,8 @@ static void prepare_packed_git(struct repository *r)\n \n \todb_prepare_alternates(r->objects);\n \tfor (source = r->objects->sources; source; source = source->next) {\n-\t\tint local = (source == r->objects->sources);\n-\t\tprepare_multi_pack_index_one(source, local);\n-\t\tprepare_packed_git_one(source, local);\n+\t\tprepare_multi_pack_index_one(source);\n+\t\tprepare_packed_git_one(source);\n \t}\n \trearrange_packed_git(r);\n \ndiff --git a/repository.c b/repository.c\nindex ecd691181f..97f0578381 100644\n--- a/repository.c\n+++ b/repository.c\n@@ -168,6 +168,7 @@ void repo_set_gitdir(struct repository *repo,\n \tif (!repo->objects->sources) {\n \t\tCALLOC_ARRAY(repo->objects->sources, 1);\n \t\trepo->objects->sources->odb = repo->objects;\n+\t\trepo->objects->sources->local = true;\n \t\trepo->objects->sources_tail = &repo->objects->sources->next;\n \t}\n \texpand_base_dir(&repo->objects->sources->path, o->object_dir,\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523936","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-2-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 02/10] odb: allow `odb_find_source()` to fail","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:42Z","receivedAt":"2025-08-11T13:46:56Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When trying to locate a source for an unknown object directory we will\ndie right away. In subsequent patches we will add new callsites though\nthat want to handle this situation gracefully instead.\n\nRefactor the function to return a `NULL` pointer if the source could not\nbe found and adapt the callsites to die instead. Introduce a new wrapper\n`odb_find_source_or_die()` that continues to die in case the source\ncould not be found.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/commit-graph.c | 4 ++--\n midx-write.c | 2 +-\n odb.c | 6 ++++++\n odb.h | 7 +++++--\n 4 files changed, 14 insertions(+), 5 deletions(-)\n\ndiff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\nindex 25018a0b9d..33fb7a5145 100644\n--- a/builtin/commit-graph.c\n+++ b/builtin/commit-graph.c\n@@ -101,7 +101,7 @@ static int graph_verify(int argc, const char **argv, const char *prefix,\n \tif (opts.progress)\n \t\tflags |= COMMIT_GRAPH_WRITE_PROGRESS;\n \n-\tsource = odb_find_source(the_repository->objects, opts.obj_dir);\n+\tsource = odb_find_source_or_die(the_repository->objects, opts.obj_dir);\n \tgraph_name = get_commit_graph_filename(source);\n \tchain_name = get_commit_graph_chain_filename(source);\n \tif (open_commit_graph(graph_name, &fd, &st))\n@@ -289,7 +289,7 @@ static int graph_write(int argc, const char **argv, const char *prefix,\n \t git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))\n \t\tflags |= COMMIT_GRAPH_WRITE_BLOOM_FILTERS;\n \n-\tsource = odb_find_source(the_repository->objects, opts.obj_dir);\n+\tsource = odb_find_source_or_die(the_repository->objects, opts.obj_dir);\n \n \tif (opts.reachable) {\n \t\tif (write_commit_graph_reachable(source, flags, &write_opts))\ndiff --git a/midx-write.c b/midx-write.c\nindex c1ae62d354..d38caceadb 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -916,7 +916,7 @@ static int write_midx_bitmap(struct write_midx_context *ctx,\n static struct multi_pack_index *lookup_multi_pack_index(struct repository *r,\n \t\t\t\t\t\t\tconst char *object_dir)\n {\n-\tstruct odb_source *source = odb_find_source(r->objects, object_dir);\n+\tstruct odb_source *source = odb_find_source_or_die(r->objects, object_dir);\n \treturn get_multi_pack_index(source);\n }\n \ndiff --git a/odb.c b/odb.c\nindex 1761a50840..4e7f14be4a 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -464,6 +464,12 @@ struct odb_source *odb_find_source(struct object_database *odb, const char *obj_\n \tfree(obj_dir_real);\n \tstrbuf_release(&odb_path_real);\n \n+\treturn source;\n+}\n+\n+struct odb_source *odb_find_source_or_die(struct object_database *odb, const char *obj_dir)\n+{\n+\tstruct odb_source *source = odb_find_source(odb, obj_dir);\n \tif (!source)\n \t\tdie(_(\"could not find object directory matching %s\"), obj_dir);\n \treturn source;\ndiff --git a/odb.h b/odb.h\nindex f9300439ba..312921077b 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -186,11 +186,14 @@ struct object_database *odb_new(struct repository *repo);\n void odb_clear(struct object_database *o);\n \n /*\n- * Find source by its object directory path. Dies in case the source couldn't\n- * be found.\n+ * Find source by its object directory path. Returns a `NULL` pointer in case\n+ * the source could not be found.\n */\n struct odb_source *odb_find_source(struct object_database *odb, const char *obj_dir);\n \n+/* Same as `odb_find_source()`, but dies in case the source doesn't exist. */\n+struct odb_source *odb_find_source_or_die(struct object_database *odb, const char *obj_dir);\n+\n /*\n * Replace the current writable object directory with the specified temporary\n * object directory; returns the former primary source.\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523937","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-3-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 03/10] odb: consistently use \"dir\" to refer to alternate's directory","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:43Z","receivedAt":"2025-08-11T13:46:59Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The functions that add an alternate object directory to the object\ndatabase are somewhat inconsistent in how they call the paramater that\nrefers to the directory path: in our headers we refer to it as \"dir\",\nwhereas in the implementation we often call it \"reference\" or \"entry\".\n\nUnify this and consistently call the parameter \"dir\". While at it,\nrefactor `link_alt_odb_entry()` to accept a C string instead of a\n`struct strbuf` as parameter to clarify that we really only need the\npath and nothing else.\n\nSuggested-by: Taylor Blau \nSigned-off-by: Patrick Steinhardt \n---\n odb.c | 30 ++++++++++++++----------------\n 1 file changed, 14 insertions(+), 16 deletions(-)\n\ndiff --git a/odb.c b/odb.c\nindex 4e7f14be4a..e41e3952ea 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -140,7 +140,7 @@ static void read_info_alternates(struct object_database *odb,\n \t\t\t\t int depth);\n \n static int link_alt_odb_entry(struct object_database *odb,\n-\t\t\t const struct strbuf *entry,\n+\t\t\t const char *dir,\n \t\t\t const char *relative_base,\n \t\t\t int depth,\n \t\t\t const char *normalized_objdir)\n@@ -151,11 +151,11 @@ static int link_alt_odb_entry(struct object_database *odb,\n \tkhiter_t pos;\n \tint ret = -1;\n \n-\tif (!is_absolute_path(entry->buf) && relative_base) {\n+\tif (!is_absolute_path(dir) && relative_base) {\n \t\tstrbuf_realpath(&pathbuf, relative_base, 1);\n \t\tstrbuf_addch(&pathbuf, '/');\n \t}\n-\tstrbuf_addbuf(&pathbuf, entry);\n+\tstrbuf_addstr(&pathbuf, dir);\n \n \tif (!strbuf_realpath(&tmp, pathbuf.buf, 0)) {\n \t\terror(_(\"unable to normalize alternate object path: %s\"),\n@@ -229,7 +229,7 @@ static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n \t\t\t\t int sep, const char *relative_base, int depth)\n {\n \tstruct strbuf objdirbuf = STRBUF_INIT;\n-\tstruct strbuf entry = STRBUF_INIT;\n+\tstruct strbuf dir = STRBUF_INIT;\n \n \tif (!alt || !*alt)\n \t\treturn;\n@@ -243,13 +243,13 @@ static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n \tstrbuf_realpath(&objdirbuf, odb->sources->path, 1);\n \n \twhile (*alt) {\n-\t\talt = parse_alt_odb_entry(alt, sep, &entry);\n-\t\tif (!entry.len)\n+\t\talt = parse_alt_odb_entry(alt, sep, &dir);\n+\t\tif (!dir.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(odb, &entry,\n+\t\tlink_alt_odb_entry(odb, dir.buf,\n \t\t\t\t relative_base, depth, objdirbuf.buf);\n \t}\n-\tstrbuf_release(&entry);\n+\tstrbuf_release(&dir);\n \tstrbuf_release(&objdirbuf);\n }\n \n@@ -273,7 +273,7 @@ static void read_info_alternates(struct object_database *odb,\n }\n \n void odb_add_to_alternates_file(struct object_database *odb,\n-\t\t\t\tconst char *reference)\n+\t\t\t\tconst char *dir)\n {\n \tstruct lock_file lock = LOCK_INIT;\n \tchar *alts = repo_git_path(odb->repo, \"objects/info/alternates\");\n@@ -290,7 +290,7 @@ void odb_add_to_alternates_file(struct object_database *odb,\n \t\tstruct strbuf line = STRBUF_INIT;\n \n \t\twhile (strbuf_getline(&line, in) != EOF) {\n-\t\t\tif (!strcmp(reference, line.buf)) {\n+\t\t\tif (!strcmp(dir, line.buf)) {\n \t\t\t\tfound = 1;\n \t\t\t\tbreak;\n \t\t\t}\n@@ -306,18 +306,17 @@ void odb_add_to_alternates_file(struct object_database *odb,\n \tif (found) {\n \t\trollback_lock_file(&lock);\n \t} else {\n-\t\tfprintf_or_die(out, \"%s\\n\", reference);\n+\t\tfprintf_or_die(out, \"%s\\n\", dir);\n \t\tif (commit_lock_file(&lock))\n \t\t\tdie_errno(_(\"unable to move new alternates file into place\"));\n \t\tif (odb->loaded_alternates)\n-\t\t\tlink_alt_odb_entries(odb, reference,\n-\t\t\t\t\t '\\n', NULL, 0);\n+\t\t\tlink_alt_odb_entries(odb, dir, '\\n', NULL, 0);\n \t}\n \tfree(alts);\n }\n \n void odb_add_to_alternates_memory(struct object_database *odb,\n-\t\t\t\t const char *reference)\n+\t\t\t\t const char *dir)\n {\n \t/*\n \t * Make sure alternates are initialized, or else our entry may be\n@@ -325,8 +324,7 @@ void odb_add_to_alternates_memory(struct object_database *odb,\n \t */\n \todb_prepare_alternates(odb);\n \n-\tlink_alt_odb_entries(odb, reference,\n-\t\t\t '\\n', NULL, 0);\n+\tlink_alt_odb_entries(odb, dir, '\\n', NULL, 0);\n }\n \n struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523938","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-4-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 04/10] odb: return newly created in-memory sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:44Z","receivedAt":"2025-08-11T13:47:02Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Callers have no trivial way to obtain the newly created object database\nsource when adding it to the in-memory list of alternates. While not yet\nneeded anywhere, a subsequent commit will want to obtain that pointer.\n\nRefactor the function to return the source to make it easily accessible.\n\nSigned-off-by: Patrick Steinhardt \n---\n odb.c | 30 ++++++++++++++++++------------\n odb.h | 4 ++--\n 2 files changed, 20 insertions(+), 14 deletions(-)\n\ndiff --git a/odb.c b/odb.c\nindex e41e3952ea..0c808bb288 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -139,17 +139,16 @@ static void read_info_alternates(struct object_database *odb,\n \t\t\t\t const char *relative_base,\n \t\t\t\t int depth);\n \n-static int link_alt_odb_entry(struct object_database *odb,\n-\t\t\t const char *dir,\n-\t\t\t const char *relative_base,\n-\t\t\t int depth,\n-\t\t\t const char *normalized_objdir)\n+static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n+\t\t\t\t\t const char *dir,\n+\t\t\t\t\t const char *relative_base,\n+\t\t\t\t\t int depth,\n+\t\t\t\t\t const char *normalized_objdir)\n {\n-\tstruct odb_source *alternate;\n+\tstruct odb_source *alternate = NULL;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n \tstruct strbuf tmp = STRBUF_INIT;\n \tkhiter_t pos;\n-\tint ret = -1;\n \n \tif (!is_absolute_path(dir) && relative_base) {\n \t\tstrbuf_realpath(&pathbuf, relative_base, 1);\n@@ -189,11 +188,11 @@ static int link_alt_odb_entry(struct object_database *odb,\n \n \t/* recursively add alternates */\n \tread_info_alternates(odb, alternate->path, depth + 1);\n-\tret = 0;\n+\n error:\n \tstrbuf_release(&tmp);\n \tstrbuf_release(&pathbuf);\n-\treturn ret;\n+\treturn alternate;\n }\n \n static const char *parse_alt_odb_entry(const char *string,\n@@ -315,16 +314,23 @@ void odb_add_to_alternates_file(struct object_database *odb,\n \tfree(alts);\n }\n \n-void odb_add_to_alternates_memory(struct object_database *odb,\n-\t\t\t\t const char *dir)\n+struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n+\t\t\t\t\t\tconst char *dir)\n {\n+\tstruct odb_source *alternate;\n+\tchar *objdir;\n+\n \t/*\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n \todb_prepare_alternates(odb);\n \n-\tlink_alt_odb_entries(odb, dir, '\\n', NULL, 0);\n+\tobjdir = real_pathdup(odb->sources->path, 1);\n+\talternate = link_alt_odb_entry(odb, dir, NULL, 0, objdir);\n+\n+\tfree(objdir);\n+\treturn alternate;\n }\n \n struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,\ndiff --git a/odb.h b/odb.h\nindex 312921077b..d769132699 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -268,8 +268,8 @@ void odb_add_to_alternates_file(struct object_database *odb,\n * recursive alternates it points to), but do not modify the on-disk alternates\n * file.\n */\n-void odb_add_to_alternates_memory(struct object_database *odb,\n-\t\t\t\t const char *dir);\n+struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n+\t\t\t\t\t\tconst char *dir);\n \n /*\n * Read an object from the database. Returns the object data and assigns object\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523939","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-5-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 05/10] odb: simplify calling `link_alt_odb_entry()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:45Z","receivedAt":"2025-08-11T13:47:05Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Callers of `link_alt_odb_entry()` are expected to pass in three\ndifferent paths:\n\n - The (potentially relative) path of the object directory that we're\n about to add.\n\n - The base that should be used to resolve a relative object directory\n path.\n\n - The resolved path to the object database's objects directory.\n\nJuggling those three paths makes the calling convention somewhat hard to\ngrok at first.\n\nAs it turns out, the third parameter is redundant: we always pass in the\nresolved path of the object database's primary source, and we already\npass in the database itself. So instead, we can resolve that path in the\nfunction itself.\n\nOne downside of this is that one caller of `link_alt_odb_entry()` calls\nthis function in a loop, so we were able to resolve the directory a\nsingle time, only. But ultimately, we only ever end up with a rather\nlimited number of alternates anyway, so the extra couple of cycles we\nsave feels more like a micro optimization.\n\nRefactor the code accordingly.\n\nSigned-off-by: Patrick Steinhardt \n---\n odb.c | 25 +++++++------------------\n 1 file changed, 7 insertions(+), 18 deletions(-)\n\ndiff --git a/odb.c b/odb.c\nindex 0c808bb288..4f884e3b50 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -142,8 +142,7 @@ static void read_info_alternates(struct object_database *odb,\n static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n \t\t\t\t\t const char *dir,\n \t\t\t\t\t const char *relative_base,\n-\t\t\t\t\t int depth,\n-\t\t\t\t\t const char *normalized_objdir)\n+\t\t\t\t\t int depth)\n {\n \tstruct odb_source *alternate = NULL;\n \tstruct strbuf pathbuf = STRBUF_INIT;\n@@ -170,7 +169,10 @@ static struct odb_source *link_alt_odb_entry(struct object_database *odb,\n \twhile (pathbuf.len && pathbuf.buf[pathbuf.len - 1] == '/')\n \t\tstrbuf_setlen(&pathbuf, pathbuf.len - 1);\n \n-\tif (!alt_odb_usable(odb, &pathbuf, normalized_objdir, &pos))\n+\tstrbuf_reset(&tmp);\n+\tstrbuf_realpath(&tmp, odb->sources->path, 1);\n+\n+\tif (!alt_odb_usable(odb, &pathbuf, tmp.buf, &pos))\n \t\tgoto error;\n \n \tCALLOC_ARRAY(alternate, 1);\n@@ -227,7 +229,6 @@ static const char *parse_alt_odb_entry(const char *string,\n static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n \t\t\t\t int sep, const char *relative_base, int depth)\n {\n-\tstruct strbuf objdirbuf = STRBUF_INIT;\n \tstruct strbuf dir = STRBUF_INIT;\n \n \tif (!alt || !*alt)\n@@ -239,17 +240,13 @@ static void link_alt_odb_entries(struct object_database *odb, const char *alt,\n \t\treturn;\n \t}\n \n-\tstrbuf_realpath(&objdirbuf, odb->sources->path, 1);\n-\n \twhile (*alt) {\n \t\talt = parse_alt_odb_entry(alt, sep, &dir);\n \t\tif (!dir.len)\n \t\t\tcontinue;\n-\t\tlink_alt_odb_entry(odb, dir.buf,\n-\t\t\t\t relative_base, depth, objdirbuf.buf);\n+\t\tlink_alt_odb_entry(odb, dir.buf, relative_base, depth);\n \t}\n \tstrbuf_release(&dir);\n-\tstrbuf_release(&objdirbuf);\n }\n \n static void read_info_alternates(struct object_database *odb,\n@@ -317,20 +314,12 @@ void odb_add_to_alternates_file(struct object_database *odb,\n struct odb_source *odb_add_to_alternates_memory(struct object_database *odb,\n \t\t\t\t\t\tconst char *dir)\n {\n-\tstruct odb_source *alternate;\n-\tchar *objdir;\n-\n \t/*\n \t * Make sure alternates are initialized, or else our entry may be\n \t * overwritten when they are.\n \t */\n \todb_prepare_alternates(odb);\n-\n-\tobjdir = real_pathdup(odb->sources->path, 1);\n-\talternate = link_alt_odb_entry(odb, dir, NULL, 0, objdir);\n-\n-\tfree(objdir);\n-\treturn alternate;\n+\treturn link_alt_odb_entry(odb, dir, NULL, 0);\n }\n \n struct odb_source *odb_set_temporary_primary_source(struct object_database *odb,\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523940","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-6-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 06/10] midx: drop redundant `struct repository` parameter","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:46Z","receivedAt":"2025-08-11T13:47:08Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"There are a couple of functions that take both a `struct repository` and\na `struct multi_pack_index`. This provides redundant information though\nwithout much benefit given that the multi-pack index already has a\npointer to its owning repository.\n\nDrop the `struct repository` parameter from such functions. While at it,\nreorder the list of parameters of `fill_midx_entry()` so that the MIDX\ncomes first to better align with our coding guidelines.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/pack-objects.c | 2 +-\n midx-write.c | 16 +++++++---------\n midx.c | 18 +++++++++---------\n midx.h | 6 +++---\n pack-bitmap.c | 4 ++--\n packfile.c | 4 ++--\n t/helper/test-read-midx.c | 4 ++--\n 7 files changed, 26 insertions(+), 28 deletions(-)\n\ndiff --git a/builtin/pack-objects.c b/builtin/pack-objects.c\nindex 3dd84495b8..b9fd685b8f 100644\n--- a/builtin/pack-objects.c\n+++ b/builtin/pack-objects.c\n@@ -1733,7 +1733,7 @@ static int want_object_in_pack_mtime(const struct object_id *oid,\n \t\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \t\tstruct pack_entry e;\n \n-\t\tif (m && fill_midx_entry(the_repository, oid, &e, m)) {\n+\t\tif (m && fill_midx_entry(m, oid, &e)) {\n \t\t\twant = want_object_in_pack_one(e.p, oid, exclude, found_pack, found_offset, found_mtime);\n \t\t\tif (want != -1)\n \t\t\t\treturn want;\ndiff --git a/midx-write.c b/midx-write.c\nindex d38caceadb..b858be475f 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -942,8 +942,7 @@ static int fill_packs_from_midx(struct write_midx_context *ctx,\n \t\t\t */\n \t\t\tif (flags & MIDX_WRITE_REV_INDEX ||\n \t\t\t preferred_pack_name) {\n-\t\t\t\tif (prepare_midx_pack(ctx->repo, m,\n-\t\t\t\t\t\t m->num_packs_in_base + i)) {\n+\t\t\t\tif (prepare_midx_pack(m, m->num_packs_in_base + i)) {\n \t\t\t\t\terror(_(\"could not load pack\"));\n \t\t\t\t\treturn 1;\n \t\t\t\t}\n@@ -1566,7 +1565,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \t\tif (count[i])\n \t\t\tcontinue;\n \n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tcontinue;\n \n \t\tif (m->packs[i]->pack_keep || m->packs[i]->is_cruft)\n@@ -1612,13 +1611,12 @@ static int compare_by_mtime(const void *a_, const void *b_)\n \treturn 0;\n }\n \n-static int want_included_pack(struct repository *r,\n-\t\t\t struct multi_pack_index *m,\n+static int want_included_pack(struct multi_pack_index *m,\n \t\t\t int pack_kept_objects,\n \t\t\t uint32_t pack_int_id)\n {\n \tstruct packed_git *p;\n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn 0;\n \tp = m->packs[pack_int_id];\n \tif (!pack_kept_objects && p->pack_keep)\n@@ -1640,7 +1638,7 @@ static void fill_included_packs_all(struct repository *r,\n \trepo_config_get_bool(r, \"repack.packkeptobjects\", &pack_kept_objects);\n \n \tfor (i = 0; i < m->num_packs; i++) {\n-\t\tif (!want_included_pack(r, m, pack_kept_objects, i))\n+\t\tif (!want_included_pack(m, pack_kept_objects, i))\n \t\t\tcontinue;\n \n \t\tinclude_pack[i] = 1;\n@@ -1664,7 +1662,7 @@ static void fill_included_packs_batch(struct repository *r,\n \tfor (i = 0; i < m->num_packs; i++) {\n \t\tpack_info[i].pack_int_id = i;\n \n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tcontinue;\n \n \t\tpack_info[i].mtime = m->packs[i]->mtime;\n@@ -1683,7 +1681,7 @@ static void fill_included_packs_batch(struct repository *r,\n \t\tstruct packed_git *p = m->packs[pack_int_id];\n \t\tuint64_t expected_size;\n \n-\t\tif (!want_included_pack(r, m, pack_kept_objects, pack_int_id))\n+\t\tif (!want_included_pack(m, pack_kept_objects, pack_int_id))\n \t\t\tcontinue;\n \n \t\t/*\ndiff --git a/midx.c b/midx.c\nindex b9ca0915a6..8459dda8c9 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -450,9 +450,10 @@ static uint32_t midx_for_pack(struct multi_pack_index **_m,\n \treturn pack_int_id - m->num_packs_in_base;\n }\n \n-int prepare_midx_pack(struct repository *r, struct multi_pack_index *m,\n+int prepare_midx_pack(struct multi_pack_index *m,\n \t\t uint32_t pack_int_id)\n {\n+\tstruct repository *r = m->repo;\n \tstruct strbuf pack_name = STRBUF_INIT;\n \tstruct strbuf key = STRBUF_INIT;\n \tstruct packed_git *p;\n@@ -507,7 +508,7 @@ struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \n #define MIDX_CHUNK_BITMAPPED_PACKS_WIDTH (2 * sizeof(uint32_t))\n \n-int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n+int nth_bitmapped_pack(struct multi_pack_index *m,\n \t\t struct bitmapped_pack *bp, uint32_t pack_int_id)\n {\n \tuint32_t local_pack_int_id = midx_for_pack(&m, pack_int_id);\n@@ -515,7 +516,7 @@ int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n \tif (!m->chunk_bitmapped_packs)\n \t\treturn error(_(\"MIDX does not contain the BTMP chunk\"));\n \n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn error(_(\"could not load bitmapped pack %\"PRIu32), pack_int_id);\n \n \tbp->p = m->packs[local_pack_int_id];\n@@ -600,10 +601,9 @@ uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos)\n \t\t\t\t\t (off_t)pos * MIDX_CHUNK_OFFSET_WIDTH);\n }\n \n-int fill_midx_entry(struct repository *r,\n+int fill_midx_entry(struct multi_pack_index *m,\n \t\t const struct object_id *oid,\n-\t\t struct pack_entry *e,\n-\t\t struct multi_pack_index *m)\n+\t\t struct pack_entry *e)\n {\n \tuint32_t pos;\n \tuint32_t pack_int_id;\n@@ -615,7 +615,7 @@ int fill_midx_entry(struct repository *r,\n \tmidx_for_object(&m, pos);\n \tpack_int_id = nth_midxed_pack_int_id(m, pos);\n \n-\tif (prepare_midx_pack(r, m, pack_int_id))\n+\tif (prepare_midx_pack(m, pack_int_id))\n \t\treturn 0;\n \tp = m->packs[pack_int_id - m->num_packs_in_base];\n \n@@ -912,7 +912,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \t\t\t\t\t\t _(\"Looking for referenced packfiles\"),\n \t\t\t\t\t\t m->num_packs + m->num_packs_in_base);\n \tfor (i = 0; i < m->num_packs + m->num_packs_in_base; i++) {\n-\t\tif (prepare_midx_pack(r, m, i))\n+\t\tif (prepare_midx_pack(m, i))\n \t\t\tmidx_report(\"failed to load pack in position %d\", i);\n \n \t\tdisplay_progress(progress, i + 1);\n@@ -989,7 +989,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \n \t\tnth_midxed_object_oid(&oid, m, pairs[i].pos);\n \n-\t\tif (!fill_midx_entry(r, &oid, &e, m)) {\n+\t\tif (!fill_midx_entry(m, &oid, &e)) {\n \t\t\tmidx_report(_(\"failed to load pack entry for oid[%d] = %s\"),\n \t\t\t\t pairs[i].pos, oid_to_hex(&oid));\n \t\t\tcontinue;\ndiff --git a/midx.h b/midx.h\nindex 28c426a823..f7e07083e1 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -103,10 +103,10 @@ void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n struct multi_pack_index *load_multi_pack_index(struct repository *r,\n \t\t\t\t\t const char *object_dir,\n \t\t\t\t\t int local);\n-int prepare_midx_pack(struct repository *r, struct multi_pack_index *m, uint32_t pack_int_id);\n+int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);\n struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \t\t\t\t uint32_t pack_int_id);\n-int nth_bitmapped_pack(struct repository *r, struct multi_pack_index *m,\n+int nth_bitmapped_pack(struct multi_pack_index *m,\n \t\t struct bitmapped_pack *bp, uint32_t pack_int_id);\n int bsearch_one_midx(const struct object_id *oid, struct multi_pack_index *m,\n \t\t uint32_t *result);\n@@ -118,7 +118,7 @@ uint32_t nth_midxed_pack_int_id(struct multi_pack_index *m, uint32_t pos);\n struct object_id *nth_midxed_object_oid(struct object_id *oid,\n \t\t\t\t\tstruct multi_pack_index *m,\n \t\t\t\t\tuint32_t n);\n-int fill_midx_entry(struct repository *r, const struct object_id *oid, struct pack_entry *e, struct multi_pack_index *m);\n+int fill_midx_entry(struct multi_pack_index *m, const struct object_id *oid, struct pack_entry *e);\n int midx_contains_pack(struct multi_pack_index *m,\n \t\t const char *idx_or_pack_name);\n int midx_preferred_pack(struct multi_pack_index *m, uint32_t *pack_int_id);\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex d14421ee20..fb0b11ca07 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -493,7 +493,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \t}\n \n \tfor (i = 0; i < bitmap_git->midx->num_packs + bitmap_git->midx->num_packs_in_base; i++) {\n-\t\tif (prepare_midx_pack(bitmap_repo(bitmap_git), bitmap_git->midx, i)) {\n+\t\tif (prepare_midx_pack(bitmap_git->midx, i)) {\n \t\t\twarning(_(\"could not open pack %s\"),\n \t\t\t\tbitmap_git->midx->pack_names[i]);\n \t\t\tgoto cleanup;\n@@ -2466,7 +2466,7 @@ void reuse_partial_packfile_from_bitmap(struct bitmap_index *bitmap_git,\n \t\tstruct multi_pack_index *m = bitmap_git->midx;\n \t\tfor (i = 0; i < m->num_packs + m->num_packs_in_base; i++) {\n \t\t\tstruct bitmapped_pack pack;\n-\t\t\tif (nth_bitmapped_pack(r, bitmap_git->midx, &pack, i) < 0) {\n+\t\t\tif (nth_bitmapped_pack(bitmap_git->midx, &pack, i) < 0) {\n \t\t\t\twarning(_(\"unable to load pack: '%s', disabling pack-reuse\"),\n \t\t\t\t\tbitmap_git->midx->pack_names[i]);\n \t\t\t\tfree(packs);\ndiff --git a/packfile.c b/packfile.c\nindex a38544b87b..acb680966d 100644\n--- a/packfile.c\n+++ b/packfile.c\n@@ -1091,7 +1091,7 @@ struct packed_git *get_all_packs(struct repository *r)\n \t\tif (!m)\n \t\t\tcontinue;\n \t\tfor (uint32_t i = 0; i < m->num_packs + m->num_packs_in_base; i++)\n-\t\t\tprepare_midx_pack(r, m, i);\n+\t\t\tprepare_midx_pack(m, i);\n \t}\n \n \treturn r->objects->packed_git;\n@@ -2077,7 +2077,7 @@ int find_pack_entry(struct repository *r, const struct object_id *oid, struct pa\n \tprepare_packed_git(r);\n \n \tfor (struct odb_source *source = r->objects->sources; source; source = source->next)\n-\t\tif (source->midx && fill_midx_entry(r, oid, e, source->midx))\n+\t\tif (source->midx && fill_midx_entry(source->midx, oid, e))\n \t\t\treturn 1;\n \n \tif (!r->objects->packed_git)\ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex da2aa036b5..e430aa247c 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -65,7 +65,7 @@ static int read_midx_file(const char *object_dir, const char *checksum,\n \t\tfor (i = 0; i < m->num_objects; i++) {\n \t\t\tnth_midxed_object_oid(&oid, m,\n \t\t\t\t\t i + m->num_objects_in_base);\n-\t\t\tfill_midx_entry(the_repository, &oid, &e, m);\n+\t\t\tfill_midx_entry(m, &oid, &e);\n \n \t\t\tprintf(\"%s %\"PRIu64\"\\t%s\\n\",\n \t\t\t oid_to_hex(&oid), e.offset, e.p->pack_name);\n@@ -126,7 +126,7 @@ static int read_midx_bitmapped_packs(const char *object_dir)\n \t\treturn 1;\n \n \tfor (i = 0; i < midx->num_packs + midx->num_packs_in_base; i++) {\n-\t\tif (nth_bitmapped_pack(the_repository, midx, &pack, i) < 0) {\n+\t\tif (nth_bitmapped_pack(midx, &pack, i) < 0) {\n \t\t\tclose_midx(midx);\n \t\t\treturn 1;\n \t\t}\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523941","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-7-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 07/10] midx: load multi-pack indices via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:47Z","receivedAt":"2025-08-11T13:47:12Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"To load a multi-pack index the caller is expected to pass both the\nrepository and the object directory where the multi-pack index is\nlocated. While this works, this layout has a couple of downsides:\n\n - We need to pass in information reduntant with the owning source,\n namely its object directory and whether the source is local or not.\n\n - We don't have access to the source when loading the multi-pack\n index. If we had that access, we could store a pointer to the owning\n source in the MIDX and thus deduplicate some information.\n\n - Multi-pack indices are inherently specific to the object source and\n its format. With the goal of pluggable object backends in mind we\n will eventually want the backends to own the logic of reading and\n writing multi-pack indices. Making the logic work on top of object\n sources is a step into that direction.\n\nRefactor loading of multi-pack indices accordingly.\n\nThis surfaces one small problem though: git-multi-pack-index(1) and our\nMIDX test helper both know to read and write multi-pack-indices located\nin a different object directory. This issue is addressed by adding the\nuser-provided object directory as an in-memory alternate.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/multi-pack-index.c | 18 ++++++++++++--\n midx.c | 57 ++++++++++++++++++++-------------------------\n midx.h | 6 ++---\n t/helper/test-read-midx.c | 25 ++++++++++++--------\n t/t5319-multi-pack-index.sh | 8 +++----\n 5 files changed, 62 insertions(+), 52 deletions(-)\n\ndiff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c\nindex aa25b06f9d..e4a9305af3 100644\n--- a/builtin/multi-pack-index.c\n+++ b/builtin/multi-pack-index.c\n@@ -64,12 +64,20 @@ static int parse_object_dir(const struct option *opt, const char *arg,\n \tchar **value = opt->value;\n \tfree(*value);\n \tif (unset)\n-\t\t*value = xstrdup(repo_get_object_directory(the_repository));\n+\t\t*value = xstrdup(the_repository->objects->sources->path);\n \telse\n \t\t*value = real_pathdup(arg, 1);\n \treturn 0;\n }\n \n+static struct odb_source *handle_object_dir_option(struct repository *repo)\n+{\n+\tstruct odb_source *source = odb_find_source(repo->objects, opts.object_dir);\n+\tif (!source)\n+\t\tsource = odb_add_to_alternates_memory(repo->objects, opts.object_dir);\n+\treturn source;\n+}\n+\n static struct option common_opts[] = {\n \tOPT_CALLBACK(0, \"object-dir\", &opts.object_dir,\n \t N_(\"directory\"),\n@@ -157,6 +165,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_write_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(repo);\n \n \tFREE_AND_NULL(options);\n \n@@ -193,6 +202,8 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,\n \t\t\tN_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n+\n \toptions = add_common_options(builtin_multi_pack_index_verify_options);\n \n \ttrace2_cmd_mode(argv[0]);\n@@ -205,10 +216,11 @@ static int cmd_multi_pack_index_verify(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_verify_usage,\n \t\t\t\t options);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn verify_midx_file(the_repository, opts.object_dir, opts.flags);\n+\treturn verify_midx_file(source, opts.flags);\n }\n \n static int cmd_multi_pack_index_expire(int argc, const char **argv,\n@@ -233,6 +245,7 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_expire_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n@@ -265,6 +278,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_repack_usage,\n \t\t\t\t options);\n+\thandle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \ndiff --git a/midx.c b/midx.c\nindex 8459dda8c9..831a7e9b5f 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -95,11 +95,10 @@ static int midx_read_object_offsets(const unsigned char *chunk_start,\n \treturn 0;\n }\n \n-static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,\n-\t\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t\t const char *midx_name,\n-\t\t\t\t\t\t\t int local)\n+static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *source,\n+\t\t\t\t\t\t\t const char *midx_name)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct multi_pack_index *m = NULL;\n \tint fd;\n \tstruct stat st;\n@@ -129,10 +128,10 @@ static struct multi_pack_index *load_multi_pack_index_one(struct repository *r,\n \tmidx_map = xmmap(NULL, midx_size, PROT_READ, MAP_PRIVATE, fd, 0);\n \tclose(fd);\n \n-\tFLEX_ALLOC_STR(m, object_dir, object_dir);\n+\tFLEX_ALLOC_STR(m, object_dir, source->path);\n \tm->data = midx_map;\n \tm->data_len = midx_size;\n-\tm->local = local;\n+\tm->local = source->local;\n \tm->repo = r;\n \n \tm->signature = get_be32(m->data);\n@@ -297,19 +296,18 @@ static int add_midx_to_chain(struct multi_pack_index *midx,\n \treturn 1;\n }\n \n-static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n-\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t int local,\n+static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source *source,\n \t\t\t\t\t\t int fd, struct stat *st,\n \t\t\t\t\t\t int *incomplete_chain)\n {\n+\tconst struct git_hash_algo *hash_algo = source->odb->repo->hash_algo;\n \tstruct multi_pack_index *midx_chain = NULL;\n \tstruct strbuf buf = STRBUF_INIT;\n \tint valid = 1;\n \tuint32_t i, count;\n \tFILE *fp = xfdopen(fd, \"r\");\n \n-\tcount = st->st_size / (r->hash_algo->hexsz + 1);\n+\tcount = st->st_size / (hash_algo->hexsz + 1);\n \n \tfor (i = 0; i < count; i++) {\n \t\tstruct multi_pack_index *m;\n@@ -318,7 +316,7 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \t\tif (strbuf_getline_lf(&buf, fp) == EOF)\n \t\t\tbreak;\n \n-\t\tif (get_oid_hex_algop(buf.buf, &layer, r->hash_algo)) {\n+\t\tif (get_oid_hex_algop(buf.buf, &layer, hash_algo)) {\n \t\t\twarning(_(\"invalid multi-pack-index chain: line '%s' \"\n \t\t\t\t \"not a hash\"),\n \t\t\t\tbuf.buf);\n@@ -329,9 +327,9 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \t\tvalid = 0;\n \n \t\tstrbuf_reset(&buf);\n-\t\tget_split_midx_filename_ext(r->hash_algo, &buf, object_dir,\n+\t\tget_split_midx_filename_ext(hash_algo, &buf, source->path,\n \t\t\t\t\t layer.hash, MIDX_EXT_MIDX);\n-\t\tm = load_multi_pack_index_one(r, object_dir, buf.buf, local);\n+\t\tm = load_multi_pack_index_one(source, buf.buf);\n \n \t\tif (m) {\n \t\t\tif (add_midx_to_chain(m, midx_chain)) {\n@@ -354,40 +352,35 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct repository *r,\n \treturn midx_chain;\n }\n \n-static struct multi_pack_index *load_multi_pack_index_chain(struct repository *r,\n-\t\t\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t\t\t int local)\n+static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *source)\n {\n \tstruct strbuf chain_file = STRBUF_INIT;\n \tstruct stat st;\n \tint fd;\n \tstruct multi_pack_index *m = NULL;\n \n-\tget_midx_chain_filename(&chain_file, object_dir);\n-\tif (open_multi_pack_index_chain(r->hash_algo, chain_file.buf, &fd, &st)) {\n+\tget_midx_chain_filename(&chain_file, source->path);\n+\tif (open_multi_pack_index_chain(source->odb->repo->hash_algo, chain_file.buf, &fd, &st)) {\n \t\tint incomplete;\n \t\t/* ownership of fd is taken over by load function */\n-\t\tm = load_midx_chain_fd_st(r, object_dir, local, fd, &st,\n-\t\t\t\t\t &incomplete);\n+\t\tm = load_midx_chain_fd_st(source, fd, &st, &incomplete);\n \t}\n \n \tstrbuf_release(&chain_file);\n \treturn m;\n }\n \n-struct multi_pack_index *load_multi_pack_index(struct repository *r,\n-\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t int local)\n+struct multi_pack_index *load_multi_pack_index(struct odb_source *source)\n {\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tstruct multi_pack_index *m;\n \n-\tget_midx_filename(r->hash_algo, &midx_name, object_dir);\n+\tget_midx_filename(source->odb->repo->hash_algo, &midx_name,\n+\t\t\t source->path);\n \n-\tm = load_multi_pack_index_one(r, object_dir,\n-\t\t\t\t midx_name.buf, local);\n+\tm = load_multi_pack_index_one(source, midx_name.buf);\n \tif (!m)\n-\t\tm = load_multi_pack_index_chain(r, object_dir, local);\n+\t\tm = load_multi_pack_index_chain(source);\n \n \tstrbuf_release(&midx_name);\n \n@@ -734,8 +727,7 @@ int prepare_multi_pack_index_one(struct odb_source *source)\n \tif (source->midx)\n \t\treturn 1;\n \n-\tsource->midx = load_multi_pack_index(r, source->path,\n-\t\t\t\t\t source->local);\n+\tsource->midx = load_multi_pack_index(source);\n \n \treturn !!source->midx;\n }\n@@ -880,12 +872,13 @@ static int compare_pair_pos_vs_id(const void *_a, const void *_b)\n \t\t\tdisplay_progress(progress, _n); \\\n \t} while (0)\n \n-int verify_midx_file(struct repository *r, const char *object_dir, unsigned flags)\n+int verify_midx_file(struct odb_source *source, unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct pair_pos_vs_id *pairs = NULL;\n \tuint32_t i;\n \tstruct progress *progress = NULL;\n-\tstruct multi_pack_index *m = load_multi_pack_index(r, object_dir, 1);\n+\tstruct multi_pack_index *m = load_multi_pack_index(source);\n \tstruct multi_pack_index *curr;\n \tverify_midx_error = 0;\n \n@@ -894,7 +887,7 @@ int verify_midx_file(struct repository *r, const char *object_dir, unsigned flag\n \t\tstruct stat sb;\n \t\tstruct strbuf filename = STRBUF_INIT;\n \n-\t\tget_midx_filename(r->hash_algo, &filename, object_dir);\n+\t\tget_midx_filename(r->hash_algo, &filename, source->path);\n \n \t\tif (!stat(filename.buf, &sb)) {\n \t\t\terror(_(\"multi-pack-index file exists, but failed to parse\"));\ndiff --git a/midx.h b/midx.h\nindex f7e07083e1..970d043989 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -100,9 +100,7 @@ void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n \t\t\t\t struct strbuf *buf, const char *object_dir,\n \t\t\t\t const unsigned char *hash, const char *ext);\n \n-struct multi_pack_index *load_multi_pack_index(struct repository *r,\n-\t\t\t\t\t const char *object_dir,\n-\t\t\t\t\t int local);\n+struct multi_pack_index *load_multi_pack_index(struct odb_source *source);\n int prepare_midx_pack(struct multi_pack_index *m, uint32_t pack_int_id);\n struct packed_git *nth_midxed_pack(struct multi_pack_index *m,\n \t\t\t\t uint32_t pack_int_id);\n@@ -136,7 +134,7 @@ int write_midx_file_only(struct repository *r, const char *object_dir,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags);\n void clear_midx_file(struct repository *r);\n-int verify_midx_file(struct repository *r, const char *object_dir, unsigned flags);\n+int verify_midx_file(struct odb_source *source, unsigned flags);\n int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags);\n int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags);\n \ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex e430aa247c..bcb8ea7671 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -11,14 +11,24 @@\n #include \"gettext.h\"\n #include \"pack-revindex.h\"\n \n+static struct multi_pack_index *setup_midx(const char *object_dir)\n+{\n+\tstruct odb_source *source;\n+\tsetup_git_directory();\n+\tsource = odb_find_source(the_repository->objects, object_dir);\n+\tif (!source)\n+\t\tsource = odb_add_to_alternates_memory(the_repository->objects,\n+\t\t\t\t\t\t object_dir);\n+\treturn load_multi_pack_index(source);\n+}\n+\n static int read_midx_file(const char *object_dir, const char *checksum,\n \t\t\t int show_objects)\n {\n \tuint32_t i;\n \tstruct multi_pack_index *m;\n \n-\tsetup_git_directory();\n-\tm = load_multi_pack_index(the_repository, object_dir, 1);\n+\tm = setup_midx(object_dir);\n \n \tif (!m)\n \t\treturn 1;\n@@ -81,8 +91,7 @@ static int read_midx_checksum(const char *object_dir)\n {\n \tstruct multi_pack_index *m;\n \n-\tsetup_git_directory();\n-\tm = load_multi_pack_index(the_repository, object_dir, 1);\n+\tm = setup_midx(object_dir);\n \tif (!m)\n \t\treturn 1;\n \tprintf(\"%s\\n\", hash_to_hex(get_midx_checksum(m)));\n@@ -96,9 +105,7 @@ static int read_midx_preferred_pack(const char *object_dir)\n \tstruct multi_pack_index *midx = NULL;\n \tuint32_t preferred_pack;\n \n-\tsetup_git_directory();\n-\n-\tmidx = load_multi_pack_index(the_repository, object_dir, 1);\n+\tmidx = setup_midx(object_dir);\n \tif (!midx)\n \t\treturn 1;\n \n@@ -119,9 +126,7 @@ static int read_midx_bitmapped_packs(const char *object_dir)\n \tstruct bitmapped_pack pack;\n \tuint32_t i;\n \n-\tsetup_git_directory();\n-\n-\tmidx = load_multi_pack_index(the_repository, object_dir, 1);\n+\tmidx = setup_midx(object_dir);\n \tif (!midx)\n \t\treturn 1;\n \ndiff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh\nindex bd75dea950..4e5e882989 100755\n--- a/t/t5319-multi-pack-index.sh\n+++ b/t/t5319-multi-pack-index.sh\n@@ -28,11 +28,11 @@ midx_read_expect () {\n \t\tEOF\n \t\tif test $NUM_PACKS -ge 1\n \t\tthen\n-\t\t\tls $OBJECT_DIR/pack/ | grep idx | sort\n+\t\t\tls \"$OBJECT_DIR\"/pack/ | grep idx | sort\n \t\tfi &&\n \t\tprintf \"object-dir: $OBJECT_DIR\\n\"\n \t} >expect &&\n-\ttest-tool read-midx $OBJECT_DIR >actual &&\n+\ttest-tool read-midx \"$OBJECT_DIR\" >actual &&\n \ttest_cmp expect actual\n }\n \n@@ -305,7 +305,7 @@ test_expect_success 'midx picks objects from preferred pack' '\n \n \t\tofs=$(git show-index expect &&\n \t\tgrep ^$b out >actual &&\n \n@@ -639,7 +639,7 @@ test_expect_success 'force some 64-bit offsets with pack-objects' '\n \t\t( cd ../objects64 && pwd ) >.git/objects/info/alternates &&\n \t\tmidx64=$(git multi-pack-index --object-dir=../objects64 write)\n \t) &&\n-\tmidx_read_expect 1 63 5 objects64 \" large-offsets\"\n+\tmidx_read_expect 1 63 5 \"$(pwd)/objects64\" \" large-offsets\"\n '\n \n test_expect_success 'verify multi-pack-index with 64-bit offsets' '\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523942","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-8-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 08/10] midx: write multi-pack indices via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:48Z","receivedAt":"2025-08-11T13:47:15Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Similar to the preceding commit, refactor the writing side of multi-pack\nindices so that we pass in the object database source where the index\nshould be written to.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/multi-pack-index.c | 19 +++++++------\n builtin/repack.c | 2 +-\n midx-write.c | 67 +++++++++++++++++++++-------------------------\n midx.h | 8 +++---\n 4 files changed, 47 insertions(+), 49 deletions(-)\n\ndiff --git a/builtin/multi-pack-index.c b/builtin/multi-pack-index.c\nindex e4a9305af3..b1e971e535 100644\n--- a/builtin/multi-pack-index.c\n+++ b/builtin/multi-pack-index.c\n@@ -147,6 +147,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \t\t\t N_(\"refs snapshot for selecting bitmap commits\")),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n \tint ret;\n \n \topts.flags |= MIDX_WRITE_BITMAP_HASH_CACHE;\n@@ -165,7 +166,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_write_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(repo);\n+\tsource = handle_object_dir_option(repo);\n \n \tFREE_AND_NULL(options);\n \n@@ -174,7 +175,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \n \t\tread_packs_from_stdin(&packs);\n \n-\t\tret = write_midx_file_only(repo, opts.object_dir, &packs,\n+\t\tret = write_midx_file_only(source, &packs,\n \t\t\t\t\t opts.preferred_pack,\n \t\t\t\t\t opts.refs_snapshot, opts.flags);\n \n@@ -185,7 +186,7 @@ static int cmd_multi_pack_index_write(int argc, const char **argv,\n \n \t}\n \n-\tret = write_midx_file(repo, opts.object_dir, opts.preferred_pack,\n+\tret = write_midx_file(source, opts.preferred_pack,\n \t\t\t opts.refs_snapshot, opts.flags);\n \n \tfree(opts.refs_snapshot);\n@@ -233,6 +234,8 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \t\t\tN_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n+\n \toptions = add_common_options(builtin_multi_pack_index_expire_options);\n \n \ttrace2_cmd_mode(argv[0]);\n@@ -245,11 +248,11 @@ static int cmd_multi_pack_index_expire(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_expire_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(the_repository);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn expire_midx_packs(the_repository, opts.object_dir, opts.flags);\n+\treturn expire_midx_packs(source, opts.flags);\n }\n \n static int cmd_multi_pack_index_repack(int argc, const char **argv,\n@@ -264,6 +267,7 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \t\t N_(\"force progress reporting\"), MIDX_PROGRESS),\n \t\tOPT_END(),\n \t};\n+\tstruct odb_source *source;\n \n \toptions = add_common_options(builtin_multi_pack_index_repack_options);\n \n@@ -278,12 +282,11 @@ static int cmd_multi_pack_index_repack(int argc, const char **argv,\n \tif (argc)\n \t\tusage_with_options(builtin_multi_pack_index_repack_usage,\n \t\t\t\t options);\n-\thandle_object_dir_option(the_repository);\n+\tsource = handle_object_dir_option(the_repository);\n \n \tFREE_AND_NULL(options);\n \n-\treturn midx_repack(the_repository, opts.object_dir,\n-\t\t\t (size_t)opts.batch_size, opts.flags);\n+\treturn midx_repack(source, (size_t)opts.batch_size, opts.flags);\n }\n \n int cmd_multi_pack_index(int argc,\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 21723866b9..94dec26f18 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -1711,7 +1711,7 @@ int cmd_repack(int argc,\n \t\tunsigned flags = 0;\n \t\tif (git_env_bool(GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL, 0))\n \t\t\tflags |= MIDX_WRITE_INCREMENTAL;\n-\t\twrite_midx_file(the_repository, repo_get_object_directory(the_repository),\n+\t\twrite_midx_file(the_repository->objects->sources,\n \t\t\t\tNULL, NULL, flags);\n \t}\n \ndiff --git a/midx-write.c b/midx-write.c\nindex b858be475f..bf7c01d4b1 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -913,13 +913,6 @@ static int write_midx_bitmap(struct write_midx_context *ctx,\n \treturn ret;\n }\n \n-static struct multi_pack_index *lookup_multi_pack_index(struct repository *r,\n-\t\t\t\t\t\t\tconst char *object_dir)\n-{\n-\tstruct odb_source *source = odb_find_source_or_die(r->objects, object_dir);\n-\treturn get_multi_pack_index(source);\n-}\n-\n static int fill_packs_from_midx(struct write_midx_context *ctx,\n \t\t\t\tconst char *preferred_pack_name, uint32_t flags)\n {\n@@ -1010,7 +1003,7 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \treturn ret;\n }\n \n-static void clear_midx_files(struct repository *r, const char *object_dir,\n+static void clear_midx_files(struct odb_source *source,\n \t\t\t const char **hashes, uint32_t hashes_nr,\n \t\t\t unsigned incremental)\n {\n@@ -1029,16 +1022,16 @@ static void clear_midx_files(struct repository *r, const char *object_dir,\n \tuint32_t i, j;\n \n \tfor (i = 0; i < ARRAY_SIZE(exts); i++) {\n-\t\tclear_incremental_midx_files_ext(object_dir, exts[i],\n+\t\tclear_incremental_midx_files_ext(source->path, exts[i],\n \t\t\t\t\t\t hashes, hashes_nr);\n \t\tfor (j = 0; j < hashes_nr; j++)\n-\t\t\tclear_midx_files_ext(object_dir, exts[i], hashes[j]);\n+\t\t\tclear_midx_files_ext(source->path, exts[i], hashes[j]);\n \t}\n \n \tif (incremental)\n-\t\tget_midx_filename(r->hash_algo, &buf, object_dir);\n+\t\tget_midx_filename(source->odb->repo->hash_algo, &buf, source->path);\n \telse\n-\t\tget_midx_chain_filename(&buf, object_dir);\n+\t\tget_midx_chain_filename(&buf, source->path);\n \n \tif (unlink(buf.buf) && errno != ENOENT)\n \t\tdie_errno(_(\"failed to clear multi-pack-index at %s\"), buf.buf);\n@@ -1046,13 +1039,14 @@ static void clear_midx_files(struct repository *r, const char *object_dir,\n \tstrbuf_release(&buf);\n }\n \n-static int write_midx_internal(struct repository *r, const char *object_dir,\n+static int write_midx_internal(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t struct string_list *packs_to_drop,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot,\n \t\t\t unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tunsigned char midx_hash[GIT_MAX_RAWSZ];\n \tuint32_t i, start_pack;\n@@ -1076,15 +1070,15 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (ctx.incremental)\n \t\tstrbuf_addf(&midx_name,\n \t\t\t \"%s/pack/multi-pack-index.d/tmp_midx_XXXXXX\",\n-\t\t\t object_dir);\n+\t\t\t source->path);\n \telse\n-\t\tget_midx_filename(r->hash_algo, &midx_name, object_dir);\n+\t\tget_midx_filename(r->hash_algo, &midx_name, source->path);\n \tif (safe_create_leading_directories(r, midx_name.buf))\n \t\tdie_errno(_(\"unable to create leading directories of %s\"),\n \t\t\t midx_name.buf);\n \n \tif (!packs_to_include || ctx.incremental) {\n-\t\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\t\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \t\tif (m && !midx_checksum_valid(m)) {\n \t\t\twarning(_(\"ignoring existing multi-pack-index; checksum mismatch\"));\n \t\t\tm = NULL;\n@@ -1138,7 +1132,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \n \tctx.to_include = packs_to_include;\n \n-\tfor_each_file_in_pack_dir(object_dir, add_pack_to_midx, &ctx);\n+\tfor_each_file_in_pack_dir(source->path, add_pack_to_midx, &ctx);\n \tstop_progress(&ctx.progress);\n \n \tif ((ctx.m && ctx.nr == ctx.m->num_packs + ctx.m->num_packs_in_base) &&\n@@ -1158,7 +1152,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\t\t * corresponding bitmap (or one wasn't requested).\n \t\t\t */\n \t\t\tif (!want_bitmap)\n-\t\t\t\tclear_midx_files_ext(object_dir, \"bitmap\", NULL);\n+\t\t\t\tclear_midx_files_ext(source->path, \"bitmap\", NULL);\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -1326,7 +1320,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (ctx.incremental) {\n \t\tstruct strbuf lock_name = STRBUF_INIT;\n \n-\t\tget_midx_chain_filename(&lock_name, object_dir);\n+\t\tget_midx_chain_filename(&lock_name, source->path);\n \t\thold_lock_file_for_update(&lk, lock_name.buf, LOCK_DIE_ON_ERROR);\n \t\tstrbuf_release(&lock_name);\n \n@@ -1389,7 +1383,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \n \tif (flags & MIDX_WRITE_REV_INDEX &&\n \t git_env_bool(\"GIT_TEST_MIDX_WRITE_REV\", 0))\n-\t\twrite_midx_reverse_index(&ctx, object_dir, midx_hash);\n+\t\twrite_midx_reverse_index(&ctx, source->path, midx_hash);\n \n \tif (flags & MIDX_WRITE_BITMAP) {\n \t\tstruct packing_data pdata;\n@@ -1412,7 +1406,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\tFREE_AND_NULL(ctx.entries);\n \t\tctx.entries_nr = 0;\n \n-\t\tif (write_midx_bitmap(&ctx, object_dir,\n+\t\tif (write_midx_bitmap(&ctx, source->path,\n \t\t\t\t midx_hash, &pdata, commits, commits_nr,\n \t\t\t\t flags) < 0) {\n \t\t\terror(_(\"could not write multi-pack bitmap\"));\n@@ -1446,7 +1440,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \t\t\treturn -1;\n \n \t\tget_split_midx_filename_ext(r->hash_algo, &final_midx_name,\n-\t\t\t\t\t object_dir, midx_hash, MIDX_EXT_MIDX);\n+\t\t\t\t\t source->path, midx_hash, MIDX_EXT_MIDX);\n \n \t\tif (rename_tempfile(&incr, final_midx_name.buf) < 0) {\n \t\t\terror_errno(_(\"unable to rename new multi-pack-index layer\"));\n@@ -1479,7 +1473,7 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \tif (commit_lock_file(&lk) < 0)\n \t\tdie_errno(_(\"could not write multi-pack-index\"));\n \n-\tclear_midx_files(r, object_dir, keep_hashes,\n+\tclear_midx_files(source, keep_hashes,\n \t\t\t ctx.num_multi_pack_indexes_before + 1,\n \t\t\t ctx.incremental);\n \n@@ -1508,29 +1502,29 @@ static int write_midx_internal(struct repository *r, const char *object_dir,\n \treturn result;\n }\n \n-int write_midx_file(struct repository *r, const char *object_dir,\n+int write_midx_file(struct odb_source *source,\n \t\t const char *preferred_pack_name,\n \t\t const char *refs_snapshot, unsigned flags)\n {\n-\treturn write_midx_internal(r, object_dir, NULL, NULL,\n+\treturn write_midx_internal(source, NULL, NULL,\n \t\t\t\t preferred_pack_name, refs_snapshot,\n \t\t\t\t flags);\n }\n \n-int write_midx_file_only(struct repository *r, const char *object_dir,\n+int write_midx_file_only(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags)\n {\n-\treturn write_midx_internal(r, object_dir, packs_to_include, NULL,\n+\treturn write_midx_internal(source, packs_to_include, NULL,\n \t\t\t\t preferred_pack_name, refs_snapshot, flags);\n }\n \n-int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags)\n+int expire_midx_packs(struct odb_source *source, unsigned flags)\n {\n \tuint32_t i, *count, result = 0;\n \tstruct string_list packs_to_drop = STRING_LIST_INIT_DUP;\n-\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \tstruct progress *progress = NULL;\n \n \tif (!m)\n@@ -1543,7 +1537,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \n \tif (flags & MIDX_PROGRESS)\n \t\tprogress = start_delayed_progress(\n-\t\t\t\t\t r,\n+\t\t\t\t\t source->odb->repo,\n \t\t\t\t\t _(\"Counting referenced objects\"),\n \t\t\t\t\t m->num_objects);\n \tfor (i = 0; i < m->num_objects; i++) {\n@@ -1555,7 +1549,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \n \tif (flags & MIDX_PROGRESS)\n \t\tprogress = start_delayed_progress(\n-\t\t\t\t\t r,\n+\t\t\t\t\t source->odb->repo,\n \t\t\t\t\t _(\"Finding and deleting unreferenced packfiles\"),\n \t\t\t\t\t m->num_packs);\n \tfor (i = 0; i < m->num_packs; i++) {\n@@ -1583,7 +1577,7 @@ int expire_midx_packs(struct repository *r, const char *object_dir, unsigned fla\n \tfree(count);\n \n \tif (packs_to_drop.nr)\n-\t\tresult = write_midx_internal(r, object_dir, NULL,\n+\t\tresult = write_midx_internal(source, NULL,\n \t\t\t\t\t &packs_to_drop, NULL, NULL, flags);\n \n \tstring_list_clear(&packs_to_drop, 0);\n@@ -1708,14 +1702,15 @@ static void fill_included_packs_batch(struct repository *r,\n \tfree(pack_info);\n }\n \n-int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags)\n+int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags)\n {\n+\tstruct repository *r = source->odb->repo;\n \tint result = 0;\n \tuint32_t i, packs_to_repack = 0;\n \tunsigned char *include_pack;\n \tstruct child_process cmd = CHILD_PROCESS_INIT;\n \tFILE *cmd_in;\n-\tstruct multi_pack_index *m = lookup_multi_pack_index(r, object_dir);\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \n \t/*\n \t * When updating the default for these configuration\n@@ -1749,7 +1744,7 @@ int midx_repack(struct repository *r, const char *object_dir, size_t batch_size,\n \n \tstrvec_push(&cmd.args, \"pack-objects\");\n \n-\tstrvec_pushf(&cmd.args, \"%s/pack/pack\", object_dir);\n+\tstrvec_pushf(&cmd.args, \"%s/pack/pack\", source->path);\n \n \tif (delta_base_offset)\n \t\tstrvec_push(&cmd.args, \"--delta-base-offset\");\n@@ -1790,7 +1785,7 @@ int midx_repack(struct repository *r, const char *object_dir, size_t batch_size,\n \t\tgoto cleanup;\n \t}\n \n-\tresult = write_midx_internal(r, object_dir, NULL, NULL, NULL, NULL,\n+\tresult = write_midx_internal(source, NULL, NULL, NULL, NULL,\n \t\t\t\t flags);\n \n cleanup:\ndiff --git a/midx.h b/midx.h\nindex 970d043989..d162001fbb 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -126,17 +126,17 @@ int prepare_multi_pack_index_one(struct odb_source *source);\n * Variant of write_midx_file which writes a MIDX containing only the packs\n * specified in packs_to_include.\n */\n-int write_midx_file(struct repository *r, const char *object_dir,\n+int write_midx_file(struct odb_source *source,\n \t\t const char *preferred_pack_name, const char *refs_snapshot,\n \t\t unsigned flags);\n-int write_midx_file_only(struct repository *r, const char *object_dir,\n+int write_midx_file_only(struct odb_source *source,\n \t\t\t struct string_list *packs_to_include,\n \t\t\t const char *preferred_pack_name,\n \t\t\t const char *refs_snapshot, unsigned flags);\n void clear_midx_file(struct repository *r);\n int verify_midx_file(struct odb_source *source, unsigned flags);\n-int expire_midx_packs(struct repository *r, const char *object_dir, unsigned flags);\n-int midx_repack(struct repository *r, const char *object_dir, size_t batch_size, unsigned flags);\n+int expire_midx_packs(struct odb_source *source, unsigned flags);\n+int midx_repack(struct odb_source *source, size_t batch_size, unsigned flags);\n \n void close_midx(struct multi_pack_index *m);\n \n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523943","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-9-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 09/10] midx: stop duplicating info redundant with its owning source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:49Z","receivedAt":"2025-08-11T13:47:18Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Multi-pack indices store some information that is redundant with their\nowning source:\n\n - The locality bit that tracks whether the source is the primary\n object source or an alternate.\n\n - The object directory path the multi-pack index is located in.\n\n - The pointer to the owning parent directory.\n\nAll of this information is already contained in `struct odb_source`. So\nnow that we always have that struct available when loading a multi-pack\nindex we have it readily accessible.\n\nDrop the redundant information and instead store a pointer to the object\nsource.\n\nSigned-off-by: Patrick Steinhardt \n---\n builtin/repack.c | 5 +++--\n midx-write.c | 9 +++++----\n midx.c | 21 +++++++++++----------\n midx.h | 7 ++-----\n pack-bitmap.c | 13 +++++++------\n pack-revindex.c | 14 +++++++-------\n t/helper/test-read-midx.c | 2 +-\n 7 files changed, 36 insertions(+), 35 deletions(-)\n\ndiff --git a/builtin/repack.c b/builtin/repack.c\nindex 94dec26f18..5af3e27357 100644\n--- a/builtin/repack.c\n+++ b/builtin/repack.c\n@@ -223,9 +223,10 @@ static void mark_packs_for_deletion(struct existing_packs *existing,\n static void remove_redundant_pack(const char *dir_name, const char *base_name)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n-\tstruct multi_pack_index *m = get_multi_pack_index(the_repository->objects->sources);\n+\tstruct odb_source *source = the_repository->objects->sources;\n+\tstruct multi_pack_index *m = get_multi_pack_index(source);\n \tstrbuf_addf(&buf, \"%s.pack\", base_name);\n-\tif (m && m->local && midx_contains_pack(m, buf.buf))\n+\tif (m && source->local && midx_contains_pack(m, buf.buf))\n \t\tclear_midx_file(the_repository);\n \tstrbuf_insertf(&buf, 0, \"%s/\", dir_name);\n \tunlink_pack_path(buf.buf, 1);\ndiff --git a/midx-write.c b/midx-write.c\nindex bf7c01d4b1..84f76856d6 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -981,10 +981,11 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \tfor (i = 0; i < ARRAY_SIZE(midx_exts); i++) {\n \t\tconst unsigned char *hash = get_midx_checksum(m);\n \n-\t\tget_midx_filename_ext(m->repo->hash_algo, &from, m->object_dir,\n+\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &from,\n+\t\t\t\t m->source->path,\n \t\t\t\t hash, midx_exts[i].non_split);\n-\t\tget_split_midx_filename_ext(m->repo->hash_algo, &to,\n-\t\t\t\t\t m->object_dir, hash,\n+\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &to,\n+\t\t\t\t\t m->source->path, hash,\n \t\t\t\t\t midx_exts[i].split);\n \n \t\tif (link(from.buf, to.buf) < 0 && errno != ENOENT) {\n@@ -1109,7 +1110,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\tif (flags & MIDX_WRITE_BITMAP && load_midx_revindex(m)) {\n \t\t\t\terror(_(\"could not load reverse index for MIDX %s\"),\n \t\t\t\t hash_to_hex_algop(get_midx_checksum(m),\n-\t\t\t\t\t\t\tm->repo->hash_algo));\n+\t\t\t\t\t\t\tm->source->odb->repo->hash_algo));\n \t\t\t\tresult = 1;\n \t\t\t\tgoto cleanup;\n \t\t\t}\ndiff --git a/midx.c b/midx.c\nindex 831a7e9b5f..81bf3c4d5f 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -26,7 +26,7 @@ int cmp_idx_or_pack_name(const char *idx_or_pack_name,\n \n const unsigned char *get_midx_checksum(struct multi_pack_index *m)\n {\n-\treturn m->data + m->data_len - m->repo->hash_algo->rawsz;\n+\treturn m->data + m->data_len - m->source->odb->repo->hash_algo->rawsz;\n }\n \n void get_midx_filename(const struct git_hash_algo *hash_algo,\n@@ -128,11 +128,10 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou\n \tmidx_map = xmmap(NULL, midx_size, PROT_READ, MAP_PRIVATE, fd, 0);\n \tclose(fd);\n \n-\tFLEX_ALLOC_STR(m, object_dir, source->path);\n+\tCALLOC_ARRAY(m, 1);\n \tm->data = midx_map;\n \tm->data_len = midx_size;\n-\tm->local = source->local;\n-\tm->repo = r;\n+\tm->source = source;\n \n \tm->signature = get_be32(m->data);\n \tif (m->signature != MIDX_SIGNATURE)\n@@ -446,7 +445,7 @@ static uint32_t midx_for_pack(struct multi_pack_index **_m,\n int prepare_midx_pack(struct multi_pack_index *m,\n \t\t uint32_t pack_int_id)\n {\n-\tstruct repository *r = m->repo;\n+\tstruct repository *r = m->source->odb->repo;\n \tstruct strbuf pack_name = STRBUF_INIT;\n \tstruct strbuf key = STRBUF_INIT;\n \tstruct packed_git *p;\n@@ -458,7 +457,7 @@ int prepare_midx_pack(struct multi_pack_index *m,\n \tif (m->packs[pack_int_id])\n \t\treturn 0;\n \n-\tstrbuf_addf(&pack_name, \"%s/pack/%s\", m->object_dir,\n+\tstrbuf_addf(&pack_name, \"%s/pack/%s\", m->source->path,\n \t\t m->pack_names[pack_int_id]);\n \n \t/* pack_map holds the \".pack\" name, but we have the .idx */\n@@ -469,7 +468,8 @@ int prepare_midx_pack(struct multi_pack_index *m,\n \t\t\t\t\tstrhash(key.buf), key.buf,\n \t\t\t\t\tstruct packed_git, packmap_ent);\n \tif (!p) {\n-\t\tp = add_packed_git(r, pack_name.buf, pack_name.len, m->local);\n+\t\tp = add_packed_git(r, pack_name.buf, pack_name.len,\n+\t\t\t\t m->source->local);\n \t\tif (p) {\n \t\t\tinstall_packed_git(r, p);\n \t\t\tlist_add_tail(&p->mru, &r->objects->packed_git_mru);\n@@ -528,7 +528,8 @@ int bsearch_one_midx(const struct object_id *oid, struct multi_pack_index *m,\n \t\t uint32_t *result)\n {\n \tint ret = bsearch_hash(oid->hash, m->chunk_oid_fanout,\n-\t\t\t m->chunk_oid_lookup, m->repo->hash_algo->rawsz,\n+\t\t\t m->chunk_oid_lookup,\n+\t\t\t m->source->odb->repo->hash_algo->rawsz,\n \t\t\t result);\n \tif (result)\n \t\t*result += m->num_objects_in_base;\n@@ -559,7 +560,7 @@ struct object_id *nth_midxed_object_oid(struct object_id *oid,\n \tn = midx_for_object(&m, n);\n \n \toidread(oid, m->chunk_oid_lookup + st_mult(m->hash_len, n),\n-\t\tm->repo->hash_algo);\n+\t\tm->source->odb->repo->hash_algo);\n \treturn oid;\n }\n \n@@ -734,7 +735,7 @@ int prepare_multi_pack_index_one(struct odb_source *source)\n \n int midx_checksum_valid(struct multi_pack_index *m)\n {\n-\treturn hashfile_checksum_valid(m->repo->hash_algo,\n+\treturn hashfile_checksum_valid(m->source->odb->repo->hash_algo,\n \t\t\t\t m->data, m->data_len);\n }\n \ndiff --git a/midx.h b/midx.h\nindex d162001fbb..71dbdec66e 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -35,6 +35,8 @@ struct odb_source;\n \t\"GIT_TEST_MULTI_PACK_INDEX_WRITE_INCREMENTAL\"\n \n struct multi_pack_index {\n+\tstruct odb_source *source;\n+\n \tconst unsigned char *data;\n \tsize_t data_len;\n \n@@ -50,7 +52,6 @@ struct multi_pack_index {\n \tuint32_t num_objects;\n \tint preferred_pack_idx;\n \n-\tint local;\n \tint has_chain;\n \n \tconst unsigned char *chunk_pack_names;\n@@ -71,10 +72,6 @@ struct multi_pack_index {\n \n \tconst char **pack_names;\n \tstruct packed_git **packs;\n-\n-\tstruct repository *repo;\n-\n-\tchar object_dir[FLEX_ARRAY];\n };\n \n #define MIDX_PROGRESS (1 << 0)\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex fb0b11ca07..01e14c34bd 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -216,7 +216,7 @@ static uint32_t bitmap_num_objects(struct bitmap_index *index)\n static struct repository *bitmap_repo(struct bitmap_index *bitmap_git)\n {\n \tif (bitmap_is_midx(bitmap_git))\n-\t\treturn bitmap_git->midx->repo;\n+\t\treturn bitmap_git->midx->source->odb->repo;\n \treturn bitmap_git->pack->repo;\n }\n \n@@ -418,13 +418,13 @@ char *midx_bitmap_filename(struct multi_pack_index *midx)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n \tif (midx->has_chain)\n-\t\tget_split_midx_filename_ext(midx->repo->hash_algo, &buf,\n-\t\t\t\t\t midx->object_dir,\n+\t\tget_split_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t\t midx->source->path,\n \t\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t\t MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(midx->repo->hash_algo, &buf,\n-\t\t\t\t midx->object_dir, get_midx_checksum(midx),\n+\t\tget_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t midx->source->path, get_midx_checksum(midx),\n \t\t\t\t MIDX_EXT_BITMAP);\n \n \treturn strbuf_detach(&buf, NULL);\n@@ -463,7 +463,8 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tget_midx_filename(midx->repo->hash_algo, &buf, midx->object_dir);\n+\t\tget_midx_filename(midx->source->odb->repo->hash_algo, &buf,\n+\t\t\t\t midx->source->path);\n \t\ttrace2_data_string(\"bitmap\", bitmap_repo(bitmap_git),\n \t\t\t\t \"ignoring extra midx bitmap file\", buf.buf);\n \t\tclose(fd);\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex 0cc422a1e6..b206518dcb 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -379,25 +379,25 @@ int load_midx_revindex(struct multi_pack_index *m)\n \t\t * not want to accidentally call munmap() in the middle of the\n \t\t * MIDX.\n \t\t */\n-\t\ttrace2_data_string(\"load_midx_revindex\", m->repo,\n+\t\ttrace2_data_string(\"load_midx_revindex\", m->source->odb->repo,\n \t\t\t\t \"source\", \"midx\");\n \t\tm->revindex_data = (const uint32_t *)m->chunk_revindex;\n \t\treturn 0;\n \t}\n \n-\ttrace2_data_string(\"load_midx_revindex\", m->repo,\n+\ttrace2_data_string(\"load_midx_revindex\", m->source->odb->repo,\n \t\t\t \"source\", \"rev\");\n \n \tif (m->has_chain)\n-\t\tget_split_midx_filename_ext(m->repo->hash_algo, &revindex_name,\n-\t\t\t\t\t m->object_dir, get_midx_checksum(m),\n+\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n+\t\t\t\t\t m->source->path, get_midx_checksum(m),\n \t\t\t\t\t MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(m->repo->hash_algo, &revindex_name,\n-\t\t\t\t m->object_dir, get_midx_checksum(m),\n+\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n+\t\t\t\t m->source->path, get_midx_checksum(m),\n \t\t\t\t MIDX_EXT_REV);\n \n-\tret = load_revindex_from_disk(m->repo->hash_algo,\n+\tret = load_revindex_from_disk(m->source->odb->repo->hash_algo,\n \t\t\t\t revindex_name.buf,\n \t\t\t\t m->num_objects,\n \t\t\t\t &m->revindex_map,\ndiff --git a/t/helper/test-read-midx.c b/t/helper/test-read-midx.c\nindex bcb8ea7671..6de5d1665a 100644\n--- a/t/helper/test-read-midx.c\n+++ b/t/helper/test-read-midx.c\n@@ -66,7 +66,7 @@ static int read_midx_file(const char *object_dir, const char *checksum,\n \tfor (i = 0; i < m->num_packs; i++)\n \t\tprintf(\"%s\\n\", m->pack_names[i]);\n \n-\tprintf(\"object-dir: %s\\n\", m->object_dir);\n+\tprintf(\"object-dir: %s\\n\", m->source->path);\n \n \tif (show_objects) {\n \t\tstruct object_id oid;\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"523944","messageId":"20250811-b4-pks-midx-deduplicate-source-info-v3-10-e442bdf2b4ad@pks.im","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"[PATCH v3 10/10] midx: compute paths via their source","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-08-11T13:46:50Z","receivedAt":"2025-08-11T13:47:21Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"With the preceding commits we started to always have the object database\nsource available when we load, write or access multi-pack indices. With\nthis in place we can change how MIDX paths are computed so that we don't\nhave to pass in the combination of a hash algorithm and object directory\nanymore, but only the object database source.\n\nRefactor the code accordingly.\n\nSigned-off-by: Patrick Steinhardt \n---\n midx-write.c | 52 ++++++++++++++++++++++++----------------------------\n midx.c | 54 +++++++++++++++++++++++++-----------------------------\n midx.h | 13 +++++--------\n pack-bitmap.c | 10 ++++------\n pack-revindex.c | 8 ++++----\n 5 files changed, 62 insertions(+), 75 deletions(-)\n\ndiff --git a/midx-write.c b/midx-write.c\nindex 84f76856d6..1dcdf3dc0f 100644\n--- a/midx-write.c\n+++ b/midx-write.c\n@@ -26,9 +26,9 @@\n #define MIDX_CHUNK_LARGE_OFFSET_WIDTH (sizeof(uint64_t))\n \n extern int midx_checksum_valid(struct multi_pack_index *m);\n-extern void clear_midx_files_ext(const char *object_dir, const char *ext,\n+extern void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t const char *keep_hash);\n-extern void clear_incremental_midx_files_ext(const char *object_dir,\n+extern void clear_incremental_midx_files_ext(struct odb_source *source,\n \t\t\t\t\t const char *ext,\n \t\t\t\t\t const char **keep_hashes,\n \t\t\t\t\t uint32_t hashes_nr);\n@@ -112,6 +112,7 @@ struct write_midx_context {\n \tstruct string_list *to_include;\n \n \tstruct repository *repo;\n+\tstruct odb_source *source;\n };\n \n static int should_include_pack(const struct write_midx_context *ctx,\n@@ -648,7 +649,6 @@ static uint32_t *midx_pack_order(struct write_midx_context *ctx)\n }\n \n static void write_midx_reverse_index(struct write_midx_context *ctx,\n-\t\t\t\t const char *object_dir,\n \t\t\t\t unsigned char *midx_hash)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n@@ -657,11 +657,10 @@ static void write_midx_reverse_index(struct write_midx_context *ctx,\n \ttrace2_region_enter(\"midx\", \"write_midx_reverse_index\", ctx->repo);\n \n \tif (ctx->incremental)\n-\t\tget_split_midx_filename_ext(ctx->repo->hash_algo, &buf,\n-\t\t\t\t\t object_dir, midx_hash,\n-\t\t\t\t\t MIDX_EXT_REV);\n+\t\tget_split_midx_filename_ext(ctx->source, &buf,\n+\t\t\t\t\t midx_hash, MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(ctx->repo->hash_algo, &buf, object_dir,\n+\t\tget_midx_filename_ext(ctx->source, &buf,\n \t\t\t\t midx_hash, MIDX_EXT_REV);\n \n \ttmp_file = write_rev_file_order(ctx->repo, NULL, ctx->pack_order,\n@@ -836,7 +835,6 @@ static struct commit **find_commits_for_midx_bitmap(uint32_t *indexed_commits_nr\n }\n \n static int write_midx_bitmap(struct write_midx_context *ctx,\n-\t\t\t const char *object_dir,\n \t\t\t const unsigned char *midx_hash,\n \t\t\t struct packing_data *pdata,\n \t\t\t struct commit **commits,\n@@ -852,12 +850,11 @@ static int write_midx_bitmap(struct write_midx_context *ctx,\n \ttrace2_region_enter(\"midx\", \"write_midx_bitmap\", ctx->repo);\n \n \tif (ctx->incremental)\n-\t\tget_split_midx_filename_ext(ctx->repo->hash_algo, &bitmap_name,\n-\t\t\t\t\t object_dir, midx_hash,\n-\t\t\t\t\t MIDX_EXT_BITMAP);\n+\t\tget_split_midx_filename_ext(ctx->source, &bitmap_name,\n+\t\t\t\t\t midx_hash, MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(ctx->repo->hash_algo, &bitmap_name,\n-\t\t\t\t object_dir, midx_hash, MIDX_EXT_BITMAP);\n+\t\tget_midx_filename_ext(ctx->source, &bitmap_name,\n+\t\t\t\t midx_hash, MIDX_EXT_BITMAP);\n \n \tif (flags & MIDX_WRITE_BITMAP_HASH_CACHE)\n \t\toptions |= BITMAP_OPT_HASH_CACHE;\n@@ -981,11 +978,9 @@ static int link_midx_to_chain(struct multi_pack_index *m)\n \tfor (i = 0; i < ARRAY_SIZE(midx_exts); i++) {\n \t\tconst unsigned char *hash = get_midx_checksum(m);\n \n-\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &from,\n-\t\t\t\t m->source->path,\n+\t\tget_midx_filename_ext(m->source, &from,\n \t\t\t\t hash, midx_exts[i].non_split);\n-\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &to,\n-\t\t\t\t\t m->source->path, hash,\n+\t\tget_split_midx_filename_ext(m->source, &to, hash,\n \t\t\t\t\t midx_exts[i].split);\n \n \t\tif (link(from.buf, to.buf) < 0 && errno != ENOENT) {\n@@ -1023,16 +1018,16 @@ static void clear_midx_files(struct odb_source *source,\n \tuint32_t i, j;\n \n \tfor (i = 0; i < ARRAY_SIZE(exts); i++) {\n-\t\tclear_incremental_midx_files_ext(source->path, exts[i],\n+\t\tclear_incremental_midx_files_ext(source, exts[i],\n \t\t\t\t\t\t hashes, hashes_nr);\n \t\tfor (j = 0; j < hashes_nr; j++)\n-\t\t\tclear_midx_files_ext(source->path, exts[i], hashes[j]);\n+\t\t\tclear_midx_files_ext(source, exts[i], hashes[j]);\n \t}\n \n \tif (incremental)\n-\t\tget_midx_filename(source->odb->repo->hash_algo, &buf, source->path);\n+\t\tget_midx_filename(source, &buf);\n \telse\n-\t\tget_midx_chain_filename(&buf, source->path);\n+\t\tget_midx_chain_filename(source, &buf);\n \n \tif (unlink(buf.buf) && errno != ENOENT)\n \t\tdie_errno(_(\"failed to clear multi-pack-index at %s\"), buf.buf);\n@@ -1065,6 +1060,7 @@ static int write_midx_internal(struct odb_source *source,\n \ttrace2_region_enter(\"midx\", \"write_midx_internal\", r);\n \n \tctx.repo = r;\n+\tctx.source = source;\n \n \tctx.incremental = !!(flags & MIDX_WRITE_INCREMENTAL);\n \n@@ -1073,7 +1069,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\t \"%s/pack/multi-pack-index.d/tmp_midx_XXXXXX\",\n \t\t\t source->path);\n \telse\n-\t\tget_midx_filename(r->hash_algo, &midx_name, source->path);\n+\t\tget_midx_filename(source, &midx_name);\n \tif (safe_create_leading_directories(r, midx_name.buf))\n \t\tdie_errno(_(\"unable to create leading directories of %s\"),\n \t\t\t midx_name.buf);\n@@ -1153,7 +1149,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\t\t * corresponding bitmap (or one wasn't requested).\n \t\t\t */\n \t\t\tif (!want_bitmap)\n-\t\t\t\tclear_midx_files_ext(source->path, \"bitmap\", NULL);\n+\t\t\t\tclear_midx_files_ext(source, \"bitmap\", NULL);\n \t\t\tgoto cleanup;\n \t\t}\n \t}\n@@ -1321,7 +1317,7 @@ static int write_midx_internal(struct odb_source *source,\n \tif (ctx.incremental) {\n \t\tstruct strbuf lock_name = STRBUF_INIT;\n \n-\t\tget_midx_chain_filename(&lock_name, source->path);\n+\t\tget_midx_chain_filename(source, &lock_name);\n \t\thold_lock_file_for_update(&lk, lock_name.buf, LOCK_DIE_ON_ERROR);\n \t\tstrbuf_release(&lock_name);\n \n@@ -1384,7 +1380,7 @@ static int write_midx_internal(struct odb_source *source,\n \n \tif (flags & MIDX_WRITE_REV_INDEX &&\n \t git_env_bool(\"GIT_TEST_MIDX_WRITE_REV\", 0))\n-\t\twrite_midx_reverse_index(&ctx, source->path, midx_hash);\n+\t\twrite_midx_reverse_index(&ctx, midx_hash);\n \n \tif (flags & MIDX_WRITE_BITMAP) {\n \t\tstruct packing_data pdata;\n@@ -1407,7 +1403,7 @@ static int write_midx_internal(struct odb_source *source,\n \t\tFREE_AND_NULL(ctx.entries);\n \t\tctx.entries_nr = 0;\n \n-\t\tif (write_midx_bitmap(&ctx, source->path,\n+\t\tif (write_midx_bitmap(&ctx,\n \t\t\t\t midx_hash, &pdata, commits, commits_nr,\n \t\t\t\t flags) < 0) {\n \t\t\terror(_(\"could not write multi-pack bitmap\"));\n@@ -1440,8 +1436,8 @@ static int write_midx_internal(struct odb_source *source,\n \t\tif (link_midx_to_chain(ctx.base_midx) < 0)\n \t\t\treturn -1;\n \n-\t\tget_split_midx_filename_ext(r->hash_algo, &final_midx_name,\n-\t\t\t\t\t source->path, midx_hash, MIDX_EXT_MIDX);\n+\t\tget_split_midx_filename_ext(source, &final_midx_name,\n+\t\t\t\t\t midx_hash, MIDX_EXT_MIDX);\n \n \t\tif (rename_tempfile(&incr, final_midx_name.buf) < 0) {\n \t\t\terror_errno(_(\"unable to rename new multi-pack-index layer\"));\ndiff --git a/midx.c b/midx.c\nindex 81bf3c4d5f..7726c13d7e 100644\n--- a/midx.c\n+++ b/midx.c\n@@ -16,9 +16,9 @@\n #define MIDX_PACK_ERROR ((void *)(intptr_t)-1)\n \n int midx_checksum_valid(struct multi_pack_index *m);\n-void clear_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t const char *keep_hash);\n-void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t char **keep_hashes,\n \t\t\t\t uint32_t hashes_nr);\n int cmp_idx_or_pack_name(const char *idx_or_pack_name,\n@@ -29,19 +29,17 @@ const unsigned char *get_midx_checksum(struct multi_pack_index *m)\n \treturn m->data + m->data_len - m->source->odb->repo->hash_algo->rawsz;\n }\n \n-void get_midx_filename(const struct git_hash_algo *hash_algo,\n-\t\t struct strbuf *out, const char *object_dir)\n+void get_midx_filename(struct odb_source *source, struct strbuf *out)\n {\n-\tget_midx_filename_ext(hash_algo, out, object_dir, NULL, NULL);\n+\tget_midx_filename_ext(source, out, NULL, NULL);\n }\n \n-void get_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t struct strbuf *out, const char *object_dir,\n+void get_midx_filename_ext(struct odb_source *source, struct strbuf *out,\n \t\t\t const unsigned char *hash, const char *ext)\n {\n-\tstrbuf_addf(out, \"%s/pack/multi-pack-index\", object_dir);\n+\tstrbuf_addf(out, \"%s/pack/multi-pack-index\", source->path);\n \tif (ext)\n-\t\tstrbuf_addf(out, \"-%s.%s\", hash_to_hex_algop(hash, hash_algo), ext);\n+\t\tstrbuf_addf(out, \"-%s.%s\", hash_to_hex_algop(hash, source->odb->repo->hash_algo), ext);\n }\n \n static int midx_read_oid_fanout(const unsigned char *chunk_start,\n@@ -222,24 +220,23 @@ static struct multi_pack_index *load_multi_pack_index_one(struct odb_source *sou\n \treturn NULL;\n }\n \n-void get_midx_chain_dirname(struct strbuf *buf, const char *object_dir)\n+void get_midx_chain_dirname(struct odb_source *source, struct strbuf *buf)\n {\n-\tstrbuf_addf(buf, \"%s/pack/multi-pack-index.d\", object_dir);\n+\tstrbuf_addf(buf, \"%s/pack/multi-pack-index.d\", source->path);\n }\n \n-void get_midx_chain_filename(struct strbuf *buf, const char *object_dir)\n+void get_midx_chain_filename(struct odb_source *source, struct strbuf *buf)\n {\n-\tget_midx_chain_dirname(buf, object_dir);\n+\tget_midx_chain_dirname(source, buf);\n \tstrbuf_addstr(buf, \"/multi-pack-index-chain\");\n }\n \n-void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t\t struct strbuf *buf, const char *object_dir,\n+void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,\n \t\t\t\t const unsigned char *hash, const char *ext)\n {\n-\tget_midx_chain_dirname(buf, object_dir);\n+\tget_midx_chain_dirname(source, buf);\n \tstrbuf_addf(buf, \"/multi-pack-index-%s.%s\",\n-\t\t hash_to_hex_algop(hash, hash_algo), ext);\n+\t\t hash_to_hex_algop(hash, source->odb->repo->hash_algo), ext);\n }\n \n static int open_multi_pack_index_chain(const struct git_hash_algo *hash_algo,\n@@ -326,7 +323,7 @@ static struct multi_pack_index *load_midx_chain_fd_st(struct odb_source *source,\n \t\tvalid = 0;\n \n \t\tstrbuf_reset(&buf);\n-\t\tget_split_midx_filename_ext(hash_algo, &buf, source->path,\n+\t\tget_split_midx_filename_ext(source, &buf,\n \t\t\t\t\t layer.hash, MIDX_EXT_MIDX);\n \t\tm = load_multi_pack_index_one(source, buf.buf);\n \n@@ -358,7 +355,7 @@ static struct multi_pack_index *load_multi_pack_index_chain(struct odb_source *s\n \tint fd;\n \tstruct multi_pack_index *m = NULL;\n \n-\tget_midx_chain_filename(&chain_file, source->path);\n+\tget_midx_chain_filename(source, &chain_file);\n \tif (open_multi_pack_index_chain(source->odb->repo->hash_algo, chain_file.buf, &fd, &st)) {\n \t\tint incomplete;\n \t\t/* ownership of fd is taken over by load function */\n@@ -374,8 +371,7 @@ struct multi_pack_index *load_multi_pack_index(struct odb_source *source)\n \tstruct strbuf midx_name = STRBUF_INIT;\n \tstruct multi_pack_index *m;\n \n-\tget_midx_filename(source->odb->repo->hash_algo, &midx_name,\n-\t\t\t source->path);\n+\tget_midx_filename(source, &midx_name);\n \n \tm = load_multi_pack_index_one(source, midx_name.buf);\n \tif (!m)\n@@ -762,7 +758,7 @@ static void clear_midx_file_ext(const char *full_path, size_t full_path_len UNUS\n \t\tdie_errno(_(\"failed to remove %s\"), full_path);\n }\n \n-void clear_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t const char *keep_hash)\n {\n \tstruct clear_midx_data data;\n@@ -776,7 +772,7 @@ void clear_midx_files_ext(const char *object_dir, const char *ext,\n \t}\n \tdata.ext = ext;\n \n-\tfor_each_file_in_pack_dir(object_dir,\n+\tfor_each_file_in_pack_dir(source->path,\n \t\t\t\t clear_midx_file_ext,\n \t\t\t\t &data);\n \n@@ -785,7 +781,7 @@ void clear_midx_files_ext(const char *object_dir, const char *ext,\n \tfree(data.keep);\n }\n \n-void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n+void clear_incremental_midx_files_ext(struct odb_source *source, const char *ext,\n \t\t\t\t char **keep_hashes,\n \t\t\t\t uint32_t hashes_nr)\n {\n@@ -801,7 +797,7 @@ void clear_incremental_midx_files_ext(const char *object_dir, const char *ext,\n \tdata.keep_nr = hashes_nr;\n \tdata.ext = ext;\n \n-\tfor_each_file_in_pack_subdir(object_dir, \"multi-pack-index.d\",\n+\tfor_each_file_in_pack_subdir(source->path, \"multi-pack-index.d\",\n \t\t\t\t clear_midx_file_ext, &data);\n \n \tfor (i = 0; i < hashes_nr; i++)\n@@ -813,7 +809,7 @@ void clear_midx_file(struct repository *r)\n {\n \tstruct strbuf midx = STRBUF_INIT;\n \n-\tget_midx_filename(r->hash_algo, &midx, r->objects->sources->path);\n+\tget_midx_filename(r->objects->sources, &midx);\n \n \tif (r->objects) {\n \t\tstruct odb_source *source;\n@@ -828,8 +824,8 @@ void clear_midx_file(struct repository *r)\n \tif (remove_path(midx.buf))\n \t\tdie(_(\"failed to clear multi-pack-index at %s\"), midx.buf);\n \n-\tclear_midx_files_ext(r->objects->sources->path, MIDX_EXT_BITMAP, NULL);\n-\tclear_midx_files_ext(r->objects->sources->path, MIDX_EXT_REV, NULL);\n+\tclear_midx_files_ext(r->objects->sources, MIDX_EXT_BITMAP, NULL);\n+\tclear_midx_files_ext(r->objects->sources, MIDX_EXT_REV, NULL);\n \n \tstrbuf_release(&midx);\n }\n@@ -888,7 +884,7 @@ int verify_midx_file(struct odb_source *source, unsigned flags)\n \t\tstruct stat sb;\n \t\tstruct strbuf filename = STRBUF_INIT;\n \n-\t\tget_midx_filename(r->hash_algo, &filename, source->path);\n+\t\tget_midx_filename(source, &filename);\n \n \t\tif (!stat(filename.buf, &sb)) {\n \t\t\terror(_(\"multi-pack-index file exists, but failed to parse\"));\ndiff --git a/midx.h b/midx.h\nindex 71dbdec66e..e241d2d690 100644\n--- a/midx.h\n+++ b/midx.h\n@@ -86,15 +86,12 @@ struct multi_pack_index {\n #define MIDX_EXT_MIDX \"midx\"\n \n const unsigned char *get_midx_checksum(struct multi_pack_index *m);\n-void get_midx_filename(const struct git_hash_algo *hash_algo,\n-\t\t struct strbuf *out, const char *object_dir);\n-void get_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t struct strbuf *out, const char *object_dir,\n+void get_midx_filename(struct odb_source *source, struct strbuf *out);\n+void get_midx_filename_ext(struct odb_source *source, struct strbuf *out,\n \t\t\t const unsigned char *hash, const char *ext);\n-void get_midx_chain_dirname(struct strbuf *buf, const char *object_dir);\n-void get_midx_chain_filename(struct strbuf *buf, const char *object_dir);\n-void get_split_midx_filename_ext(const struct git_hash_algo *hash_algo,\n-\t\t\t\t struct strbuf *buf, const char *object_dir,\n+void get_midx_chain_dirname(struct odb_source *source, struct strbuf *out);\n+void get_midx_chain_filename(struct odb_source *source, struct strbuf *out);\n+void get_split_midx_filename_ext(struct odb_source *source, struct strbuf *buf,\n \t\t\t\t const unsigned char *hash, const char *ext);\n \n struct multi_pack_index *load_multi_pack_index(struct odb_source *source);\ndiff --git a/pack-bitmap.c b/pack-bitmap.c\nindex 01e14c34bd..058bdb5d7d 100644\n--- a/pack-bitmap.c\n+++ b/pack-bitmap.c\n@@ -418,13 +418,12 @@ char *midx_bitmap_filename(struct multi_pack_index *midx)\n {\n \tstruct strbuf buf = STRBUF_INIT;\n \tif (midx->has_chain)\n-\t\tget_split_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t\t midx->source->path,\n+\t\tget_split_midx_filename_ext(midx->source, &buf,\n \t\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t\t MIDX_EXT_BITMAP);\n \telse\n-\t\tget_midx_filename_ext(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t midx->source->path, get_midx_checksum(midx),\n+\t\tget_midx_filename_ext(midx->source, &buf,\n+\t\t\t\t get_midx_checksum(midx),\n \t\t\t\t MIDX_EXT_BITMAP);\n \n \treturn strbuf_detach(&buf, NULL);\n@@ -463,8 +462,7 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,\n \n \tif (bitmap_git->pack || bitmap_git->midx) {\n \t\tstruct strbuf buf = STRBUF_INIT;\n-\t\tget_midx_filename(midx->source->odb->repo->hash_algo, &buf,\n-\t\t\t\t midx->source->path);\n+\t\tget_midx_filename(midx->source, &buf);\n \t\ttrace2_data_string(\"bitmap\", bitmap_repo(bitmap_git),\n \t\t\t\t \"ignoring extra midx bitmap file\", buf.buf);\n \t\tclose(fd);\ndiff --git a/pack-revindex.c b/pack-revindex.c\nindex b206518dcb..d0791cc493 100644\n--- a/pack-revindex.c\n+++ b/pack-revindex.c\n@@ -389,12 +389,12 @@ int load_midx_revindex(struct multi_pack_index *m)\n \t\t\t \"source\", \"rev\");\n \n \tif (m->has_chain)\n-\t\tget_split_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n-\t\t\t\t\t m->source->path, get_midx_checksum(m),\n+\t\tget_split_midx_filename_ext(m->source, &revindex_name,\n+\t\t\t\t\t get_midx_checksum(m),\n \t\t\t\t\t MIDX_EXT_REV);\n \telse\n-\t\tget_midx_filename_ext(m->source->odb->repo->hash_algo, &revindex_name,\n-\t\t\t\t m->source->path, get_midx_checksum(m),\n+\t\tget_midx_filename_ext(m->source, &revindex_name,\n+\t\t\t\t get_midx_checksum(m),\n \t\t\t\t MIDX_EXT_REV);\n \n \tret = load_revindex_from_disk(m->source->odb->repo->hash_algo,\n\n-- \n2.51.0.rc1.163.g2494970778.dirty\n\n"},{"id":"525163","messageId":"xmqq1povt67o.fsf@gitster.g","threadId":"63871","inReplyTo":"20250811-b4-pks-midx-deduplicate-source-info-v3-0-e442bdf2b4ad@pks.im","subject":"Re: [PATCH v3 00/10] midx: stop duplicating info redundant with their sources","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2025-08-28T22:46:35Z","receivedAt":"2025-08-28T22:46:38Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt writes:\n\n> Changes in v2:\n> - Fix a comment typo.\n> - Introduce another commit that simplifies the calling convention for\n> `link_alt_odb_entry()`.\n> - Link to v1: https://lore.kernel.org/r/20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im\n>\n> Changes in v3:\n> - Introduce `odb_find_source_or_die()` so that we don't have to repeat\n> the calls to `die()`, as suggested by Taylor.\n> - Split out a patch to adapt `link_alt_odb_entry()` and friends to\n> consistently name the parameter that refers to the alternate object\n> directory's path.\n> - Link to v2: https://lore.kernel.org/r/20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im\n\nThis has gone quiet even though the previous two iterations got\nfairly detailed review. Should we declare victory and mark the\ntopic for 'next' now?\n\nThanks.\n\n"},{"id":"525175","messageId":"aLD1s/LAcYlDujJG@nand.local","threadId":"63871","inReplyTo":"xmqq1povt67o.fsf@gitster.g","subject":"Re: [PATCH v3 00/10] midx: stop duplicating info redundant with their sources","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2025-08-29T00:34:59Z","receivedAt":"2025-08-29T00:35:02Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Aug 28, 2025 at 03:46:35PM -0700, Junio C Hamano wrote:\n> Patrick Steinhardt writes:\n>\n> > Changes in v2:\n> > - Fix a comment typo.\n> > - Introduce another commit that simplifies the calling convention for\n> > `link_alt_odb_entry()`.\n> > - Link to v1: https://lore.kernel.org/r/20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im\n> >\n> > Changes in v3:\n> > - Introduce `odb_find_source_or_die()` so that we don't have to repeat\n> > the calls to `die()`, as suggested by Taylor.\n> > - Split out a patch to adapt `link_alt_odb_entry()` and friends to\n> > consistently name the parameter that refers to the alternate object\n> > directory's path.\n> > - Link to v2: https://lore.kernel.org/r/20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im\n>\n> This has gone quiet even though the previous two iterations got\n> fairly detailed review. Should we declare victory and mark the\n> topic for 'next' now?\n\nNo objections from me; though I still would feel a little better with\nStolee's opinion on the MIDX --object-dir stuff.\n\nThanks,\nTaylor\n"},{"id":"525251","messageId":"f6c9b931-1587-4517-9cca-9e9d0899021e@gmail.com","threadId":"63871","inReplyTo":"aLD1s/LAcYlDujJG@nand.local","subject":"Re: [PATCH v3 00/10] midx: stop duplicating info redundant with their sources","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2025-08-30T13:39:02Z","receivedAt":"2025-08-30T13:39:04Z","isPatch":true,"sender":{"key":"stolee@gmail.com","avatar":"https://avatars.githubusercontent.com/u/570044?v=4"},"body":"On 8/28/25 8:34 PM, Taylor Blau wrote:\n> On Thu, Aug 28, 2025 at 03:46:35PM -0700, Junio C Hamano wrote:\n>> Patrick Steinhardt writes:\n>>\n>>> Changes in v2:\n>>> - Fix a comment typo.\n>>> - Introduce another commit that simplifies the calling convention for\n>>> `link_alt_odb_entry()`.\n>>> - Link to v1: https://lore.kernel.org/r/20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im\n>>>\n>>> Changes in v3:\n>>> - Introduce `odb_find_source_or_die()` so that we don't have to repeat\n>>> the calls to `die()`, as suggested by Taylor.\n>>> - Split out a patch to adapt `link_alt_odb_entry()` and friends to\n>>> consistently name the parameter that refers to the alternate object\n>>> directory's path.\n>>> - Link to v2: https://lore.kernel.org/r/20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im\n>>\n>> This has gone quiet even though the previous two iterations got\n>> fairly detailed review. Should we declare victory and mark the\n>> topic for 'next' now?\n> \n> No objections from me; though I still would feel a little better with\n> Stolee's opinion on the MIDX --object-dir stuff.\n\nSorry I had missed this series and in particular how it handles the\n--object-dir option.\n\nI think the modification to add structure to the --object-dir option\nby passing around an object source is going to make things better.\n\nThe one thing I was very careful about was that we are not trying\nto create a full \"repository\" struct based on the object-dir, because\nthere may not be one! We only know that there is an alternate object\ndatabase where we want to break the norm and perform writes. The\nchanges here make that seem like it continues to work.\n\nThanks,\n-Stolee\n\n"},{"id":"525291","messageId":"aLaQWPK0Kxw8o7VV@pks.im","threadId":"63871","inReplyTo":"f6c9b931-1587-4517-9cca-9e9d0899021e@gmail.com","subject":"Re: [PATCH v3 00/10] midx: stop duplicating info redundant with their sources","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2025-09-02T06:36:08Z","receivedAt":"2025-09-02T06:36:21Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Sat, Aug 30, 2025 at 09:39:02AM -0400, Derrick Stolee wrote:\n> On 8/28/25 8:34 PM, Taylor Blau wrote:\n> > On Thu, Aug 28, 2025 at 03:46:35PM -0700, Junio C Hamano wrote:\n> > > Patrick Steinhardt writes:\n> > > \n> > > > Changes in v2:\n> > > > - Fix a comment typo.\n> > > > - Introduce another commit that simplifies the calling convention for\n> > > > `link_alt_odb_entry()`.\n> > > > - Link to v1: https://lore.kernel.org/r/20250729-b4-pks-midx-deduplicate-source-info-v1-0-748db2eda3b5@pks.im\n> > > > \n> > > > Changes in v3:\n> > > > - Introduce `odb_find_source_or_die()` so that we don't have to repeat\n> > > > the calls to `die()`, as suggested by Taylor.\n> > > > - Split out a patch to adapt `link_alt_odb_entry()` and friends to\n> > > > consistently name the parameter that refers to the alternate object\n> > > > directory's path.\n> > > > - Link to v2: https://lore.kernel.org/r/20250807-b4-pks-midx-deduplicate-source-info-v2-0-bcffb8fc119c@pks.im\n> > > \n> > > This has gone quiet even though the previous two iterations got\n> > > fairly detailed review. Should we declare victory and mark the\n> > > topic for 'next' now?\n> > \n> > No objections from me; though I still would feel a little better with\n> > Stolee's opinion on the MIDX --object-dir stuff.\n> \n> Sorry I had missed this series and in particular how it handles the\n> --object-dir option.\n> \n> I think the modification to add structure to the --object-dir option\n> by passing around an object source is going to make things better.\n> \n> The one thing I was very careful about was that we are not trying\n> to create a full \"repository\" struct based on the object-dir, because\n> there may not be one! We only know that there is an alternate object\n> database where we want to break the norm and perform writes. The\n> changes here make that seem like it continues to work.\n\nYup, this should continue to work as-is indeed. We also have a couple of\ntests in t5319 which verify that this works alright. And overall I think\nthe whole logic becomes a bit easier to reason about now that we pass\naround the object source explicitly.\n\nThanks for your thoughts. With that I'm happy to have this series merged\ndown to 'next'.\n\nPatrick\n"}]}