{"thread":{"id":"66855","subject":"[PATCH 00/12] Introduce pluggable commit graphs","startedAt":"2026-11-16T13:07:36Z","lastAt":"2026-11-16T20:53:56Z","messageCount":14,"participants":["Patrick Steinhardt","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":12},"messages":[{"id":"558535","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":null,"subject":"[PATCH 00/12] Introduce pluggable commit graphs","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:36Z","receivedAt":"2026-11-16T13:07:36Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Hi,\n\nthis patch series introduces pluggable commit graphs such that an object\ndatabase backend is in full control over where exactly the commit graph\nis being read from or written to.\n\nAn alternative form of this would have been to abstract the overall\nfunctionality of a commit graph and instead expose its functionality.\nBut that would have been a much larger undertaking, so for now I decided\nto go the easier step and retain the data structure itself. But that\ndoes not preclude a subsequent step eventually to abstract the data\nstructure even further if we ever wanted to.\n\nMeanwhile, with this change all logic to derive storage paths for commit\ngraphs moves into the \"files\" backend.\n\nThis series is built on fbd2d1e765d (The 18th batch, 2026-11-09) with\nps/commit-graph-write-refactor at a9993c1000d (commit-graph: modernize\nfunction names, 2026-11-04) merged into it.\n\nThanks!\n\nPatrick\n\n---\nPatrick Steinhardt (12):\n      commit-graph: simplify linking commit graph chains\n      commit-graph: simplify reading a single commit graph\n      commit-graph: expose function to validate a commit graph chain\n      commit-graph: decide mergeable and chainable layers by depth\n      commit-graph: introduce common exit for `write_commit_graph_file()`\n      commit-graph: derive layer infos from the passed-in chain\n      commit-graph: split out check whether writing commit graphs is allowed\n      commit-graph: split out logic to write a new commit graph layer\n      commit-graph: mark incomplete commit graph chains\n      odb/source: make commit graphs pluggable\n      odb/source-files: move logic to derive commit graph paths\n      odb/source-files: internalize logic to derive commit graph paths\n\n builtin/commit-graph.c     |  87 ++++---\n builtin/fetch.c            |   4 +-\n builtin/gc.c               |   3 +-\n commit-graph.c             | 585 +++++++--------------------------------------\n commit-graph.h             |  74 +++---\n odb.c                      |   6 +\n odb.h                      |  10 +\n odb/source-files.c         | 478 ++++++++++++++++++++++++++++++++++++\n odb/source-files.h         |  12 +\n odb/source.h               |  52 ++++\n t/helper/test-read-graph.c |   3 +-\n 11 files changed, 740 insertions(+), 574 deletions(-)\n\n\n---\nbase-commit: 0c2ef2cc2023eab2fe8bc54d542fe4f93314f186\nchange-id: 20261110-pks-odb-pluggable-commit-graphs-e9dc9674f0ae\n\n\n"},{"id":"558536","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-1-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 01/12] commit-graph: simplify linking commit graph chains","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:37Z","receivedAt":"2026-11-16T13:07:37Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When we load a commit graph chain we verify for every layer that the\nhash of the base commit graph matches what the current tip has in its\nbase graphs chunk. To do so we pass both the chain itself as well as the\narray of object IDs that we have read from the commit-graph chain file\nto `add_graph_to_chain()`.\n\nBut the array of object IDs is redundant information: we have already\nloaded and linked the base layers, and as part of that we have stored\ntheir respective hashes in the structure itself. Consequently, we can\nalready derive the hashes by walking the chain of base layers.\n\nRefactor the function to only take the new tip and base layers as input\nand derive all the other information ad-hoc and make the function\npublic. This prepares for a subsequent commit, where we are about to\nmove the logic that creates commit graph chains into the object database\nbackend.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n commit-graph.c | 52 ++++++++++++++++++++++++----------------------------\n commit-graph.h |  8 ++++++++\n 2 files changed, 32 insertions(+), 28 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex da65cd3deb9..94c761907a0 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -575,50 +575,48 @@ static void validate_mixed_bloom_settings(struct commit_graph *g)\n \t}\n }\n \n-static int add_graph_to_chain(struct commit_graph *g,\n-\t\t\t      struct commit_graph *chain,\n-\t\t\t      struct object_id *oids,\n-\t\t\t      int n)\n+int commit_graph_link_base(struct commit_graph *g,\n+\t\t\t   struct commit_graph *base)\n {\n-\tstruct commit_graph *cur_g = chain;\n+\tstruct commit_graph *layer;\n+\tsize_t n = 0;\n+\n+\tfor (layer = base; layer; layer = layer->base_graph)\n+\t\tn++;\n \n \tif (n && !g->chunk_base_graphs) {\n \t\twarning(_(\"commit-graph has no base graphs chunk\"));\n-\t\treturn 0;\n+\t\treturn -1;\n \t}\n \n \tif (g->chunk_base_graphs_size / g->hash_algo->rawsz < n) {\n \t\twarning(_(\"commit-graph base graphs chunk is too small\"));\n-\t\treturn 0;\n+\t\treturn -1;\n \t}\n \n-\twhile (n) {\n+\tfor (layer = base; layer; layer = layer->base_graph) {\n \t\tn--;\n-\n-\t\tif (!cur_g ||\n-\t\t    !oideq(&oids[n], &cur_g->oid) ||\n-\t\t    !hasheq(oids[n].hash, g->chunk_base_graphs + st_mult(g->hash_algo->rawsz, n),\n+\t\tif (!hasheq(layer->oid.hash,\n+\t\t\t    g->chunk_base_graphs + st_mult(g->hash_algo->rawsz, n),\n \t\t\t    g->hash_algo)) {\n \t\t\twarning(_(\"commit-graph chain does not match\"));\n-\t\t\treturn 0;\n+\t\t\treturn -1;\n \t\t}\n-\n-\t\tcur_g = cur_g->base_graph;\n \t}\n \n-\tif (chain) {\n-\t\tif (unsigned_add_overflows(chain->num_commits,\n-\t\t\t\t\t   chain->num_commits_in_base)) {\n+\tif (base) {\n+\t\tif (unsigned_add_overflows(base->num_commits,\n+\t\t\t\t\t   base->num_commits_in_base)) {\n \t\t\twarning(_(\"commit count in base graph too high: %\"PRIuMAX),\n-\t\t\t\t(uintmax_t)chain->num_commits_in_base);\n-\t\t\treturn 0;\n+\t\t\t\t(uintmax_t)base->num_commits_in_base);\n+\t\t\treturn -1;\n \t\t}\n-\t\tg->num_commits_in_base = chain->num_commits + chain->num_commits_in_base;\n+\t\tg->num_commits_in_base = base->num_commits + base->num_commits_in_base;\n \t}\n \n-\tg->base_graph = chain;\n+\tg->base_graph = base;\n \n-\treturn 1;\n+\treturn 0;\n }\n \n int open_commit_graph_chain(const char *chain_file,\n@@ -652,20 +650,19 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n {\n \tstruct commit_graph *graph_chain = NULL;\n \tstruct strbuf line = STRBUF_INIT;\n-\tstruct object_id *oids;\n \tint i = 0, valid = 1, count;\n \tFILE *fp = xfdopen(fd, \"r\");\n \n \tcount = st->st_size / (odb->repo->hash_algo->hexsz + 1);\n-\tCALLOC_ARRAY(oids, count);\n \n \tfor (i = 0; i < count; i++) {\n \t\tstruct odb_source_files *files;\n+\t\tstruct object_id oid;\n \n \t\tif (strbuf_getline_lf(&line, fp) == EOF)\n \t\t\tbreak;\n \n-\t\tif (get_oid_hex_algop(line.buf, &oids[i], odb->repo->hash_algo)) {\n+\t\tif (get_oid_hex_algop(line.buf, &oid, odb->repo->hash_algo)) {\n \t\t\twarning(_(\"invalid commit-graph chain: line '%s' not a hash\"),\n \t\t\t\tline.buf);\n \t\t\tvalid = 0;\n@@ -682,7 +679,7 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n \t\t\tfree(graph_name);\n \n \t\t\tif (g) {\n-\t\t\t\tif (add_graph_to_chain(g, graph_chain, oids, i)) {\n+\t\t\t\tif (!commit_graph_link_base(g, graph_chain)) {\n \t\t\t\t\tgraph_chain = g;\n \t\t\t\t\tvalid = 1;\n \t\t\t\t} else {\n@@ -702,7 +699,6 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n \tvalidate_mixed_generation_chain(graph_chain);\n \tvalidate_mixed_bloom_settings(graph_chain);\n \n-\tfree(oids);\n \tfclose(fp);\n \tstrbuf_release(&line);\n \ndiff --git a/commit-graph.h b/commit-graph.h\nindex 2156499a688..f0e7cc93461 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -128,6 +128,14 @@ struct commit_graph *load_commit_graph_one_fd_st(struct repository *repo,\n struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n \t\t\t\t\t\t   int fd, struct stat *st,\n \t\t\t\t\t\t   int *incomplete_chain);\n+\n+/*\n+ * Link the commit graph layer `g` on top of the chain `base`. `base` may be\n+ * NULL if `g` is the root of a new chain. Returns 0 on success, a negative\n+ * error code otherwise.\n+ */\n+int commit_graph_link_base(struct commit_graph *g,\n+\t\t\t   struct commit_graph *base);\n struct commit_graph *read_commit_graph_one(struct repository *repo,\n \t\t\t\t\t   const char *dir);\n \n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558537","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-2-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 02/12] commit-graph: simplify reading a single commit graph","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:38Z","receivedAt":"2026-11-16T13:07:38Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In a subsequent commit we're about to move the logic to locate commit\ngraphs and commit graph chains into the object database backend. With\nthat change, the responsibility of the commit-graph subsystem changes so\nthat it only provides the functionality to read and write commit graphs,\nwhereas the backend owns the locations those are read from and written\nto.\n\nThe function `load_commit_graph_one_fd_st()` is responsible for loading\na single commit graph layer from disk. It receives as input the file\ndescriptor and the directory it is being read from. While the former\nmakes sense, the latter doesn't: we already populate the graph's file\nname in `load_commit_graph_one()`, so we populate paths in two different\nlocations.\n\nBesides that, the actual paths are about to become a responsibility of\nthe backend. So arguably, generic functionality that loads the commit\ngraph should not be exposed to it.\n\nStop passing the directory path and instead populate it in\n`load_commit_graph_one()`, which will later on be moved into the \"files\"\nbackend. While at it, rename `load_commit_graph_one_fd_st()` to\n`commit_graph_read()`.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/commit-graph.c |  2 +-\n commit-graph.c         | 15 +++++++--------\n commit-graph.h         | 12 +++++-------\n 3 files changed, 13 insertions(+), 16 deletions(-)\n\ndiff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\nindex 8194a528406..2a5a75dc22f 100644\n--- a/builtin/commit-graph.c\n+++ b/builtin/commit-graph.c\n@@ -111,7 +111,7 @@ static int cmd_commit_graph_verify(int argc,\n \tif (opened == OPENED_NONE)\n \t\treturn 0;\n \telse if (opened == OPENED_GRAPH)\n-\t\tgraph = load_commit_graph_one_fd_st(repo, dir->abspath, fd, &st);\n+\t\tgraph = commit_graph_read(repo, fd, &st);\n \telse\n \t\tgraph = load_commit_graph_chain_fd_st(repo->objects, fd, &st,\n \t\t\t\t\t\t      &incomplete_chain);\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 94c761907a0..c427a6f90f7 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -257,9 +257,8 @@ int open_commit_graph(const char *graph_file, int *fd, struct stat *st)\n \treturn 1;\n }\n \n-struct commit_graph *load_commit_graph_one_fd_st(struct repository *repo,\n-\t\t\t\t\t\t const char *dir,\n-\t\t\t\t\t\t int fd, struct stat *st)\n+struct commit_graph *commit_graph_read(struct repository *repo,\n+\t\t\t\t       int fd, struct stat *st)\n {\n \tvoid *graph_map;\n \tsize_t graph_size;\n@@ -276,9 +275,7 @@ struct commit_graph *load_commit_graph_one_fd_st(struct repository *repo,\n \tclose(fd);\n \n \tret = commit_graph_parse(repo, graph_map, graph_size);\n-\tif (ret)\n-\t\tret->dir = absolute_pathdup(dir);\n-\telse\n+\tif (!ret)\n \t\tmunmap(graph_map, graph_size);\n \n \treturn ret;\n@@ -508,9 +505,11 @@ static struct commit_graph *load_commit_graph_one(struct repository *repo,\n \tif (!open_ok)\n \t\treturn NULL;\n \n-\tg = load_commit_graph_one_fd_st(repo, dir, fd, &st);\n-\tif (g)\n+\tg = commit_graph_read(repo, fd, &st);\n+\tif (g) {\n+\t\tg->dir = absolute_pathdup(dir);\n \t\tg->filename = absolute_pathdup(graph_file);\n+\t}\n \n \treturn g;\n }\ndiff --git a/commit-graph.h b/commit-graph.h\nindex f0e7cc93461..8660c7fe698 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -117,14 +117,12 @@ struct commit_graph {\n };\n \n /*\n- * Load commit graphs from the given object directory `dir`. The directory may\n- * be given as a relative path; it is canonicalized internally so that graphs\n- * loaded from the same directory compare equal regardless of how the caller\n- * spelled the path.\n+ * Read a single commit graph layer from the given file descriptor. The file\n+ * descriptor will be closed regardless of whether loading succeeds. Returns\n+ * NULL on error.\n  */\n-struct commit_graph *load_commit_graph_one_fd_st(struct repository *repo,\n-\t\t\t\t\t\t const char *dir,\n-\t\t\t\t\t\t int fd, struct stat *st);\n+struct commit_graph *commit_graph_read(struct repository *repo,\n+\t\t\t\t       int fd, struct stat *st);\n struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n \t\t\t\t\t\t   int fd, struct stat *st,\n \t\t\t\t\t\t   int *incomplete_chain);\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558538","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-3-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 03/12] commit-graph: expose function to validate a commit graph chain","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:39Z","receivedAt":"2026-11-16T13:07:39Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When we have loaded a commit graph chain we need to validate whether all\nof its layers have been written with compatible settings. If that's not\nthe case, then some functionality of the chain needs to be restricted as\nit would otherwise yield inconsistent results.\n\nIn a subsequent commit we'll move loading of commit graphs into the\nobject database backend, and consequently the backend will have to\nperform this validation. Expose this functionality to prepare for this.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n commit-graph.c | 9 +++++++--\n commit-graph.h | 7 +++++++\n 2 files changed, 14 insertions(+), 2 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex c427a6f90f7..9464522feb5 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -574,6 +574,12 @@ static void validate_mixed_bloom_settings(struct commit_graph *g)\n \t}\n }\n \n+void commit_graph_validate_chain(struct commit_graph *g)\n+{\n+\tvalidate_mixed_generation_chain(g);\n+\tvalidate_mixed_bloom_settings(g);\n+}\n+\n int commit_graph_link_base(struct commit_graph *g,\n \t\t\t   struct commit_graph *base)\n {\n@@ -695,8 +701,7 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n \t\t}\n \t}\n \n-\tvalidate_mixed_generation_chain(graph_chain);\n-\tvalidate_mixed_bloom_settings(graph_chain);\n+\tcommit_graph_validate_chain(graph_chain);\n \n \tfclose(fp);\n \tstrbuf_release(&line);\ndiff --git a/commit-graph.h b/commit-graph.h\nindex 8660c7fe698..dc4f6194c72 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -134,6 +134,13 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n  */\n int commit_graph_link_base(struct commit_graph *g,\n \t\t\t   struct commit_graph *base);\n+\n+/*\n+ * Validate the chain of commit graph layers to ensure that all layers are\n+ * compatible with one another. This may disable generation data and Bloom\n+ * filters in case they aren't.\n+ */\n+void commit_graph_validate_chain(struct commit_graph *g);\n struct commit_graph *read_commit_graph_one(struct repository *repo,\n \t\t\t\t\t   const char *dir);\n \n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558539","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-4-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 04/12] commit-graph: decide mergeable and chainable layers by depth","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:40Z","receivedAt":"2026-11-16T13:07:40Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When writing a split commit graph, `split_graph_merge_strategy()`\ndecides which layers of the existing chain get merged into the new\nlayer and which layer the new one builds upon. This is decided on two\nsimple rules:\n\n  - A layer may be merged in case it is part of the same local object\n    directory.\n\n  - A layer may be added to the chain in case it is either a split\n    commit graph layer, regardless of whether it is local or not. Or in\n    case it is local, we may also add it to our chain in case it is a\n    non-split commit graph, in which case we convert it to a split one.\n\nThese checks require us to know about where exactly these commit graphs\nare stored on disk as we compare directories of the respective files\nwith one another to figure out whether or not a layer is local. So as a\nconsequence, this policy becomes backend-specific once the backend\ncontrols where exactly commit graphs are stored.\n\nRefactor the code so that we instead tell the merge strategy how many of\nthe topmost layers are mergeable and chainable, respectively, which will\nlet the backends decide on that property.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n commit-graph.c | 51 ++++++++++++++++++++++++++++++++++++++++-----------\n 1 file changed, 40 insertions(+), 11 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 9464522feb5..7f7146e752a 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -1151,6 +1151,8 @@ struct write_commit_graph_context {\n \tuint64_t progress_cnt;\n \n \tchar *base_graph_name;\n+\tsize_t mergeable_layers;\n+\tsize_t chainable_layers;\n \tint num_commit_graphs_before;\n \tint num_commit_graphs_after;\n \tchar **commit_graph_filenames_before;\n@@ -2283,6 +2285,7 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,\n \tenum commit_graph_split_flags flags = ctx->opts->split_flags;\n \tint max_commits = ctx->opts->max_commits;\n \tint size_mult = 2;\n+\tsize_t depth = 0;\n \tuint32_t i;\n \n \tif (ctx->opts->size_multiple)\n@@ -2299,7 +2302,7 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,\n \t    flags != COMMIT_GRAPH_SPLIT_REPLACE) {\n \t\twhile (g && (g->num_commits <= st_mult(size_mult, num_commits) ||\n \t\t\t    (max_commits && num_commits > max_commits))) {\n-\t\t\tif (strcmp(g->dir, ctx->dir))\n+\t\t\tif (depth >= ctx->mergeable_layers)\n \t\t\t\tbreak;\n \n \t\t\tif (unsigned_add_overflows(num_commits, g->num_commits))\n@@ -2309,6 +2312,7 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,\n \t\t\t\t    (uintmax_t)g->num_commits);\n \t\t\tnum_commits += g->num_commits;\n \t\t\tg = g->base_graph;\n+\t\t\tdepth++;\n \n \t\t\tctx->num_commit_graphs_after--;\n \t\t}\n@@ -2320,16 +2324,13 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,\n \t\tBUG(\"split_graph_merge_strategy: num_commit_graphs_after \"\n \t\t    \"should be 1 with --split=replace\");\n \n-\tif (ctx->num_commit_graphs_after == 2) {\n-\t\tchar *old_graph_name = get_commit_graph_filename(g->dir);\n-\n-\t\tif (!strcmp(g->filename, old_graph_name) &&\n-\t\t    strcmp(g->dir, ctx->dir)) {\n-\t\t\tctx->num_commit_graphs_after = 1;\n-\t\t\tctx->new_base_graph = NULL;\n-\t\t}\n-\n-\t\tfree(old_graph_name);\n+\t/*\n+\t * If we cannot add the new base to our chain we instead merge it and\n+\t * all of its base layers.\n+\t */\n+\tif (ctx->new_base_graph && depth >= ctx->chainable_layers) {\n+\t\tctx->num_commit_graphs_after = 1;\n+\t\tctx->new_base_graph = NULL;\n \t}\n \n \tCALLOC_ARRAY(ctx->commit_graph_filenames_after, ctx->num_commit_graphs_after);\n@@ -2560,6 +2561,7 @@ int commit_graph_write(struct repository *r,\n \tstruct oidset reachable_commits = OIDSET_INIT;\n \tstruct topo_level_slab topo_levels;\n \tstruct commit_graph *g;\n+\tbool mergeable = true;\n \n \tif (!!opts->pack_indexes + !!opts->commits + !!(opts->flags & COMMIT_GRAPH_WRITE_REACHABLE) > 1)\n \t\tBUG(\"attempting to write commit-graph with mutually incompatible options\");\n@@ -2597,6 +2599,33 @@ int commit_graph_write(struct repository *r,\n \tfor (struct commit_graph *chain = g; chain; chain = chain->base_graph)\n \t\tchain->topo_levels = &topo_levels;\n \n+\t/*\n+\t * We only consider merging layers that belong to our own object\n+\t * directory. Furthermore, we can only chain on top of either:\n+\t *\n+\t *   - A split commit graph layer, regardless of whether or not it is\n+\t *     local.\n+\t *\n+\t *   - A local non-split commit graph, in which case we'll convert that\n+\t *     layer into a split commit graph layer.\n+\t */\n+\tfor (struct commit_graph *layer = g; layer; layer = layer->base_graph) {\n+\t\tchar *path = get_commit_graph_filename(layer->dir);\n+\t\tbool is_split = !!strcmp(layer->filename, path);\n+\t\tbool is_local = !strcmp(layer->dir, ctx.dir);\n+\n+\t\tfree(path);\n+\n+\t\tif (!is_local && !is_split)\n+\t\t\tbreak;\n+\t\tif (!is_local)\n+\t\t\tmergeable = false;\n+\n+\t\tif (mergeable)\n+\t\t\tctx.mergeable_layers++;\n+\t\tctx.chainable_layers++;\n+\t}\n+\n \tif (opts->flags & COMMIT_GRAPH_WRITE_BLOOM_FILTERS)\n \t\tctx.changed_paths = 1;\n \tif (!(opts->flags & COMMIT_GRAPH_NO_WRITE_BLOOM_FILTERS)) {\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558540","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-5-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 05/12] commit-graph: introduce common exit for `write_commit_graph_file()`","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:41Z","receivedAt":"2026-11-16T13:07:41Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In the next commit we're going to refactor `write_commit_graph_file()`.\nAs part of that refactoring we'll perform a couple more allocations in\nthis function that need to be cleaned up on exit.\n\nPrepare for this by introducing a common exit path now already, as that\nwill make the diff a tiny bit easier to read.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n commit-graph.c | 34 +++++++++++++++++++---------------\n 1 file changed, 19 insertions(+), 15 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 7f7146e752a..c146f20f68f 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -2099,6 +2099,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \tstruct strbuf progress_title = STRBUF_INIT;\n \tstruct chunkfile *cf;\n \tunsigned char file_hash[GIT_MAX_RAWSZ];\n+\tint ret;\n \n \tif (ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n \t\tstruct strbuf tmp_file = STRBUF_INIT;\n@@ -2112,9 +2113,9 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \t}\n \n \tif (safe_create_leading_directories(ctx->r, ctx->graph_name)) {\n-\t\terror(_(\"unable to create leading directories of %s\"),\n-\t\t\tctx->graph_name);\n-\t\treturn -1;\n+\t\tret = error(_(\"unable to create leading directories of %s\"),\n+\t\t\t    ctx->graph_name);\n+\t\tgoto out;\n \t}\n \n \tif (ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n@@ -2126,14 +2127,14 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \n \t\tgraph_layer = mks_tempfile_m(ctx->graph_name, 0444);\n \t\tif (!graph_layer) {\n-\t\t\terror(_(\"unable to create temporary graph layer\"));\n-\t\t\treturn -1;\n+\t\t\tret = error(_(\"unable to create temporary graph layer\"));\n+\t\t\tgoto out;\n \t\t}\n \n \t\tif (adjust_shared_perm(ctx->r, get_tempfile_path(graph_layer))) {\n-\t\t\terror(_(\"unable to adjust shared permissions for '%s'\"),\n-\t\t\t      get_tempfile_path(graph_layer));\n-\t\t\treturn -1;\n+\t\t\tret = error(_(\"unable to adjust shared permissions for '%s'\"),\n+\t\t\t\t    get_tempfile_path(graph_layer));\n+\t\t\tgoto out;\n \t\t}\n \n \t\tf = hashfd(ctx->r->hash_algo,\n@@ -2227,8 +2228,8 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \t\tint result;\n \n \t\tif (!chainf) {\n-\t\t\terror(_(\"unable to open commit-graph chain file\"));\n-\t\t\treturn -1;\n+\t\t\tret = error(_(\"unable to open commit-graph chain file\"));\n+\t\t\tgoto out;\n \t\t}\n \n \t\tif (ctx->base_graph_name) {\n@@ -2243,8 +2244,8 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \t\t\t\tresult = rename(ctx->base_graph_name, dest);\n \n \t\t\t\tif (result) {\n-\t\t\t\t\terror(_(\"failed to rename base commit-graph file\"));\n-\t\t\t\t\treturn -1;\n+\t\t\t\t\tret = error(_(\"failed to rename base commit-graph file\"));\n+\t\t\t\t\tgoto out;\n \t\t\t\t}\n \t\t\t}\n \t\t} else {\n@@ -2267,14 +2268,17 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \t\t\tfprintf(get_lock_file_fp(&lk), \"%s\\n\", ctx->commit_graph_hash_after[i]);\n \n \t\tif (result) {\n-\t\t\terror(_(\"failed to rename temporary commit-graph file\"));\n-\t\t\treturn -1;\n+\t\t\tret = error(_(\"failed to rename temporary commit-graph file\"));\n+\t\t\tgoto out;\n \t\t}\n \t}\n \n \tcommit_lock_file(&lk);\n+\tret = 0;\n \n-\treturn 0;\n+out:\n+\trollback_lock_file(&lk);\n+\treturn ret;\n }\n \n static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558541","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-6-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 06/12] commit-graph: derive layer infos from the passed-in chain","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:42Z","receivedAt":"2026-11-16T13:07:42Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In addition to writing the new commit-graph layer, we also need to do a\nbit of bookkeeping when writing a split commit graph.\n\n  - We track the list of hashes that the new chain is supposed to\n    consist of.\n\n  - We track the list of filenames part of the new chain so that we\n    don't prune them.\n\n  - We track the list of filenames that will be merged so that we can\n    bump their mtime.\n\n  - We track the name of the old non-split commit-graph so that we can\n    rename it in case it is being converted into a split commit-graph.\n\nAll of this information is passed in by the caller via a couple of\narrays in the write context, but it is in fact redundant: we already\nhave all information available via the passed-in commit graph.\n\nRefactor the code accordingly. This will make it easier to move this\nlogic into the \"files\" backend as we can now cleanly split out the\npath-dependent logic from `write_commit_graph_file()`.\n\nWhile the logic should be mostly equivalent, there's one change in\nbehaviour: in the preimage, we call `expire_commit_graphs()` regardless\nof whether or not writing the new commit graph file succeeds, but now we\nonly do so on success. This is arguably an improvement though, as\nexpiring commit graphs even when we haven't written any new ones is\nsomewhat puzzling.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n commit-graph.c | 227 ++++++++++++++++++++++++---------------------------------\n 1 file changed, 97 insertions(+), 130 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex c146f20f68f..caf40fd6b01 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -25,6 +25,7 @@\n #include \"bloom.h\"\n #include \"commit-slab.h\"\n #include \"shallow.h\"\n+#include \"string-list.h\"\n #include \"json-writer.h\"\n #include \"trace2.h\"\n #include \"tree.h\"\n@@ -1150,14 +1151,10 @@ struct write_commit_graph_context {\n \tint progress_done;\n \tuint64_t progress_cnt;\n \n-\tchar *base_graph_name;\n \tsize_t mergeable_layers;\n \tsize_t chainable_layers;\n \tint num_commit_graphs_before;\n \tint num_commit_graphs_after;\n-\tchar **commit_graph_filenames_before;\n-\tchar **commit_graph_filenames_after;\n-\tchar **commit_graph_hash_after;\n \tuint32_t new_num_commits_in_base;\n \tstruct commit_graph *new_base_graph;\n \n@@ -2089,14 +2086,24 @@ static int write_graph_chunk_base(struct hashfile *f,\n \treturn 0;\n }\n \n+static void mark_commit_graphs(struct commit_graph *tip,\n+\t\t\t       struct commit_graph *base);\n+static void expire_commit_graphs(const char *dirpath,\n+\t\t\t\t struct string_list *kept,\n+\t\t\t\t timestamp_t expire_time);\n+\n static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n {\n-\tuint32_t i;\n \tstruct hashfile *f;\n-\tstruct tempfile *graph_layer; /* when COMMIT_GRAPH_WRITE_SPLIT is non-zero */\n+\tstruct tempfile *graph_layer = NULL; /* when COMMIT_GRAPH_WRITE_SPLIT is non-zero */\n \tstruct lock_file lk = LOCK_INIT;\n \tconst unsigned hashsz = ctx->r->hash_algo->rawsz;\n \tstruct strbuf progress_title = STRBUF_INIT;\n+\tstruct string_list chain_hashes = STRING_LIST_INIT_DUP;\n+\tstruct string_list kept = STRING_LIST_INIT_DUP;\n+\tstruct commit_graph *base = ctx->new_base_graph, *chain;\n+\tchar *base_rename_from = NULL, *base_rename_to = NULL;\n+\tchar *new_path = NULL;\n \tstruct chunkfile *cf;\n \tunsigned char file_hash[GIT_MAX_RAWSZ];\n \tint ret;\n@@ -2206,78 +2213,102 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \tstop_progress(&ctx->progress);\n \tstrbuf_release(&progress_title);\n \n-\tif ((ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) &&\n-\t    ctx->base_graph_name && ctx->num_commit_graphs_after > 1) {\n-\t\tchar *new_base_hash = xstrdup(oid_to_hex(&ctx->new_base_graph->oid));\n-\t\tchar *new_base_name = get_split_graph_filename(ctx->new_base_graph->dir, new_base_hash);\n+\tfinalize_hashfile(f, file_hash, FSYNC_COMPONENT_COMMIT_GRAPH,\n+\t\t\t  CSUM_HASH_IN_STREAM | CSUM_FSYNC);\n+\tfree_chunkfile(cf);\n+\n+\t/*\n+\t * Snapshot everything we need from the current chain before closing\n+\t * it, as we must not have any layers mapped when replacing them.\n+\t */\n+\tif (ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\t\tconst char *new_hash = hash_to_hex_algop(file_hash, ctx->r->hash_algo);\n+\n+\t\tnew_path = get_split_graph_filename(ctx->dir, new_hash);\n+\n+\t\tstring_list_append(&chain_hashes, new_hash);\n+\t\tfor (chain = base; chain; chain = chain->base_graph)\n+\t\t\tstring_list_append(&chain_hashes, oid_to_hex(&chain->oid));\n \n-\t\tfree(ctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 2]);\n-\t\tfree(ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 2]);\n-\t\tctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 2] = new_base_name;\n-\t\tctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 2] = new_base_hash;\n+\t\t/*\n+\t\t * In case we have a non-split commit graph we need to convert\n+\t\t * it into the base layer of the new split commit graph by\n+\t\t * renaming it.\n+\t\t */\n+\t\tif (base) {\n+\t\t\tbase_rename_to = get_split_graph_filename(base->dir, oid_to_hex(&base->oid));\n+\t\t\tif (strcmp(base->filename, base_rename_to))\n+\t\t\t\tbase_rename_from = xstrdup(base->filename);\n+\t\t\telse\n+\t\t\t\tFREE_AND_NULL(base_rename_to);\n+\t\t}\n+\n+\t\tfor (chain = base; chain; chain = chain->base_graph) {\n+\t\t\tif (chain == base && base_rename_to)\n+\t\t\t\tstring_list_append(&kept, base_rename_to);\n+\t\t\telse\n+\t\t\t\tstring_list_append(&kept, chain->filename);\n+\t\t}\n+\t\tstring_list_append(&kept, new_path);\n+\n+\t\tmark_commit_graphs(ctx->r->objects->commit_graph, base);\n \t}\n \n \tcommit_graph_close(ctx->r->objects);\n-\tfinalize_hashfile(f, file_hash, FSYNC_COMPONENT_COMMIT_GRAPH,\n-\t\t\t  CSUM_HASH_IN_STREAM | CSUM_FSYNC);\n-\tfree_chunkfile(cf);\n \n \tif (ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n \t\tFILE *chainf = fdopen_lock_file(&lk, \"w\");\n-\t\tchar *final_graph_name;\n-\t\tint result;\n \n \t\tif (!chainf) {\n \t\t\tret = error(_(\"unable to open commit-graph chain file\"));\n \t\t\tgoto out;\n \t\t}\n \n-\t\tif (ctx->base_graph_name) {\n-\t\t\tconst char *dest;\n-\t\t\tint idx = ctx->num_commit_graphs_after - 1;\n-\t\t\tif (ctx->num_commit_graphs_after > 1)\n-\t\t\t\tidx--;\n-\n-\t\t\tdest = ctx->commit_graph_filenames_after[idx];\n-\n-\t\t\tif (strcmp(ctx->base_graph_name, dest)) {\n-\t\t\t\tresult = rename(ctx->base_graph_name, dest);\n-\n-\t\t\t\tif (result) {\n-\t\t\t\t\tret = error(_(\"failed to rename base commit-graph file\"));\n-\t\t\t\t\tgoto out;\n-\t\t\t\t}\n+\t\tif (base_rename_from) {\n+\t\t\tif (rename(base_rename_from, base_rename_to)) {\n+\t\t\t\tret = error(_(\"failed to rename base commit-graph file\"));\n+\t\t\t\tgoto out;\n \t\t\t}\n-\t\t} else {\n+\t\t} else if (!base) {\n \t\t\tchar *graph_name = get_commit_graph_filename(ctx->dir);\n \t\t\tunlink(graph_name);\n \t\t\tfree(graph_name);\n \t\t}\n \n-\t\tfree(ctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1]);\n-\t\tctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1] =\n-\t\t\txstrdup(hash_to_hex_algop(file_hash, ctx->r->hash_algo));\n-\t\tfinal_graph_name = get_split_graph_filename(ctx->dir,\n-\t\t\t\t\tctx->commit_graph_hash_after[ctx->num_commit_graphs_after - 1]);\n-\t\tfree(ctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 1]);\n-\t\tctx->commit_graph_filenames_after[ctx->num_commit_graphs_after - 1] = final_graph_name;\n-\n-\t\tresult = rename_tempfile(&graph_layer, final_graph_name);\n-\n-\t\tfor (i = 0; i < ctx->num_commit_graphs_after; i++)\n-\t\t\tfprintf(get_lock_file_fp(&lk), \"%s\\n\", ctx->commit_graph_hash_after[i]);\n-\n-\t\tif (result) {\n+\t\tif (rename_tempfile(&graph_layer, new_path)) {\n \t\t\tret = error(_(\"failed to rename temporary commit-graph file\"));\n \t\t\tgoto out;\n \t\t}\n+\n+\t\t/*\n+\t\t * Chain hashes are ordered newest-first, but the chain file is\n+\t\t * oldest-first so we iterate in reverse.\n+\t\t */\n+\t\tfor (size_t i = chain_hashes.nr; i; i--)\n+\t\t\tfprintf(chainf, \"%s\\n\", chain_hashes.items[i - 1].string);\n \t}\n \n \tcommit_lock_file(&lk);\n+\n+\tif (!(ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT)) {\n+\t\tchar *chain_name = get_commit_graph_chain_filename(ctx->dir);\n+\t\tunlink(chain_name);\n+\t\tfree(chain_name);\n+\t}\n+\n+\texpire_commit_graphs(ctx->dir, &kept,\n+\t\t\t     ctx->opts->expire_time ? ctx->opts->expire_time : time(NULL));\n+\n \tret = 0;\n \n out:\n \trollback_lock_file(&lk);\n+\tdelete_tempfile(&graph_layer);\n+\tstring_list_clear(&chain_hashes, 0);\n+\tstring_list_clear(&kept, 0);\n+\tfree(base_rename_from);\n+\tfree(base_rename_to);\n+\tfree(new_path);\n \treturn ret;\n }\n \n@@ -2290,7 +2321,6 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,\n \tint max_commits = ctx->opts->max_commits;\n \tint size_mult = 2;\n \tsize_t depth = 0;\n-\tuint32_t i;\n \n \tif (ctx->opts->size_multiple)\n \t\tsize_mult = ctx->opts->size_multiple;\n@@ -2337,30 +2367,12 @@ static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,\n \t\tctx->new_base_graph = NULL;\n \t}\n \n-\tCALLOC_ARRAY(ctx->commit_graph_filenames_after, ctx->num_commit_graphs_after);\n-\tCALLOC_ARRAY(ctx->commit_graph_hash_after, ctx->num_commit_graphs_after);\n-\n-\tfor (i = 0; i < ctx->num_commit_graphs_after &&\n-\t\t    i < ctx->num_commit_graphs_before; i++)\n-\t\tctx->commit_graph_filenames_after[i] = xstrdup(ctx->commit_graph_filenames_before[i]);\n-\n-\ti = ctx->num_commit_graphs_before - 1;\n-\tg = graph_to_merge;\n-\n-\twhile (g) {\n-\t\tif (i < ctx->num_commit_graphs_after)\n-\t\t\tctx->commit_graph_hash_after[i] = xstrdup(oid_to_hex(&g->oid));\n-\n-\t\t/*\n-\t\t * If the topmost remaining layer has generation data chunk, the\n-\t\t * resultant layer also has generation data chunk.\n-\t\t */\n-\t\tif (i == ctx->num_commit_graphs_after - 2)\n-\t\t\tctx->write_generation_data = !!g->chunk_generation_data;\n-\n-\t\ti--;\n-\t\tg = g->base_graph;\n-\t}\n+\t/*\n+\t * If the topmost remaining layer has generation data chunk, the\n+\t * resultant layer also has generation data chunk.\n+\t */\n+\tif (ctx->new_base_graph)\n+\t\tctx->write_generation_data = !!ctx->new_base_graph->chunk_generation_data;\n }\n \n static void merge_commit_graph(struct write_commit_graph_context *ctx,\n@@ -2464,48 +2476,37 @@ static void merge_commit_graphs(struct write_commit_graph_context *ctx,\n \t\tctx->new_num_commits_in_base = g->num_commits + g->num_commits_in_base;\n \t}\n \n-\tif (ctx->new_base_graph)\n-\t\tctx->base_graph_name = xstrdup(ctx->new_base_graph->filename);\n-\n \tsort_and_scan_merged_commits(ctx);\n }\n \n-static void mark_commit_graphs(struct write_commit_graph_context *ctx)\n+static void mark_commit_graphs(struct commit_graph *tip,\n+\t\t\t       struct commit_graph *base)\n {\n-\tuint32_t i;\n \ttime_t now = time(NULL);\n \n-\tfor (i = ctx->num_commit_graphs_after - 1; i < ctx->num_commit_graphs_before; i++) {\n+\tfor (struct commit_graph *g = tip; g && g != base; g = g->base_graph) {\n \t\tstruct stat st;\n \t\tstruct utimbuf updated_time;\n \n-\t\tif (stat(ctx->commit_graph_filenames_before[i], &st) < 0)\n+\t\tif (stat(g->filename, &st) < 0)\n \t\t\tcontinue;\n \n \t\tupdated_time.actime = st.st_atime;\n \t\tupdated_time.modtime = now;\n-\t\tutime(ctx->commit_graph_filenames_before[i], &updated_time);\n+\t\tutime(g->filename, &updated_time);\n \t}\n }\n \n-static void expire_commit_graphs(struct write_commit_graph_context *ctx)\n+static void expire_commit_graphs(const char *dirpath,\n+\t\t\t\t struct string_list *kept,\n+\t\t\t\t timestamp_t expire_time)\n {\n \tstruct strbuf path = STRBUF_INIT;\n \tDIR *dir;\n \tstruct dirent *de;\n \tsize_t dirnamelen;\n-\ttimestamp_t expire_time = time(NULL);\n-\n-\tif (ctx->opts->expire_time)\n-\t\texpire_time = ctx->opts->expire_time;\n-\tif (!(ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT)) {\n-\t\tchar *chain_file_name = get_commit_graph_chain_filename(ctx->dir);\n-\t\tunlink(chain_file_name);\n-\t\tfree(chain_file_name);\n-\t\tctx->num_commit_graphs_after = 0;\n-\t}\n \n-\tstrbuf_addstr(&path, ctx->dir);\n+\tstrbuf_addstr(&path, dirpath);\n \tstrbuf_addstr(&path, \"/info/commit-graphs\");\n \tdir = opendir(path.buf);\n \n@@ -2516,7 +2517,6 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx)\n \tdirnamelen = path.len;\n \twhile ((de = readdir(dir)) != NULL) {\n \t\tstruct stat st;\n-\t\tuint32_t i, found = 0;\n \n \t\tstrbuf_setlen(&path, dirnamelen);\n \t\tstrbuf_addstr(&path, de->d_name);\n@@ -2529,20 +2529,12 @@ static void expire_commit_graphs(struct write_commit_graph_context *ctx)\n \t\tif (path.len < 6 || strcmp(path.buf + path.len - 6, \".graph\"))\n \t\t\tcontinue;\n \n-\t\tfor (i = 0; i < ctx->num_commit_graphs_after; i++) {\n-\t\t\tif (!strcmp(ctx->commit_graph_filenames_after[i],\n-\t\t\t\t    path.buf)) {\n-\t\t\t\tfound = 1;\n-\t\t\t\tbreak;\n-\t\t\t}\n-\t\t}\n-\n-\t\tif (!found)\n+\t\tif (!unsorted_string_list_has_string(kept, path.buf))\n \t\t\tunlink(path.buf);\n \t}\n \n out:\n-\tif(dir)\n+\tif (dir)\n \t\tclosedir(dir);\n \tstrbuf_release(&path);\n }\n@@ -2652,14 +2644,6 @@ int commit_graph_write(struct repository *r,\n \t\tfor (struct commit_graph *chain = g; chain; chain = chain->base_graph)\n \t\t\tctx.num_commit_graphs_before++;\n \n-\t\tif (ctx.num_commit_graphs_before) {\n-\t\t\tALLOC_ARRAY(ctx.commit_graph_filenames_before, ctx.num_commit_graphs_before);\n-\t\t\ti = ctx.num_commit_graphs_before;\n-\n-\t\t\tfor (struct commit_graph *chain = g; chain; chain = chain->base_graph)\n-\t\t\t\tctx.commit_graph_filenames_before[--i] = xstrdup(chain->filename);\n-\t\t}\n-\n \t\treplace = ctx.opts->split_flags & COMMIT_GRAPH_SPLIT_REPLACE;\n \t}\n \n@@ -2727,14 +2711,8 @@ int commit_graph_write(struct repository *r,\n \tif (ctx.changed_paths)\n \t\tdeinit_bloom_filters();\n \n-\tif (ctx.opts->flags & COMMIT_GRAPH_WRITE_SPLIT)\n-\t\tmark_commit_graphs(&ctx);\n-\n-\texpire_commit_graphs(&ctx);\n-\n cleanup:\n \tfree(ctx.graph_name);\n-\tfree(ctx.base_graph_name);\n \tfree(ctx.dir);\n \tcommit_stack_clear(&ctx.commits);\n \toidset_clear(&reachable_commits);\n@@ -2750,17 +2728,6 @@ int commit_graph_write(struct repository *r,\n \t\t}\n \t}\n \n-\tfor (i = 0; i < ctx.num_commit_graphs_before; i++)\n-\t\tfree(ctx.commit_graph_filenames_before[i]);\n-\tfree(ctx.commit_graph_filenames_before);\n-\n-\tfor (i = 0; i < ctx.num_commit_graphs_after; i++) {\n-\t\tfree(ctx.commit_graph_filenames_after[i]);\n-\t\tfree(ctx.commit_graph_hash_after[i]);\n-\t}\n-\tfree(ctx.commit_graph_filenames_after);\n-\tfree(ctx.commit_graph_hash_after);\n-\n \treturn res;\n }\n \n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558542","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-7-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 07/12] commit-graph: split out check whether writing commit graphs is allowed","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:43Z","receivedAt":"2026-11-16T13:07:43Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In a subsequent commit we're about to split out the logic to write a\ncommit graph into the object database backends, which will need to\ndetermine whether writing a graph is enabled in the first place. There\ncan be various reasons though why we cannot write a new graph, and we\ncertainly don't want to repeat this logic across all potential writers.\n\nExtract the logic into a new function to prepare for this.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n commit-graph.c | 37 ++++++++++++++++++++++++-------------\n commit-graph.h |  3 +++\n 2 files changed, 27 insertions(+), 13 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex caf40fd6b01..c673c0424da 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -2539,6 +2539,29 @@ static void expire_commit_graphs(const char *dirpath,\n \tstrbuf_release(&path);\n }\n \n+bool commit_graph_write_allowed(struct repository *r)\n+{\n+\tprepare_repo_settings(r);\n+\n+\tif (!r->settings.core_commit_graph) {\n+\t\twarning(_(\"attempting to write a commit-graph, but 'core.commitGraph' is disabled\"));\n+\t\treturn false;\n+\t}\n+\n+\tif (!commit_graph_compatible(r))\n+\t\treturn false;\n+\n+\tif (r->settings.commit_graph_changed_paths_version < -1 ||\n+\t    r->settings.commit_graph_changed_paths_version > 2) {\n+\t\twarning(_(\"attempting to write a commit-graph, but \"\n+\t\t\t  \"'commitGraph.changedPathsVersion' (%d) is not supported\"),\n+\t\t\tr->settings.commit_graph_changed_paths_version);\n+\t\treturn false;\n+\t}\n+\n+\treturn true;\n+}\n+\n int commit_graph_write(struct repository *r,\n \t\t       const char *dir,\n \t\t       const struct commit_graph_write_options *opts)\n@@ -2562,20 +2585,8 @@ int commit_graph_write(struct repository *r,\n \tif (!!opts->pack_indexes + !!opts->commits + !!(opts->flags & COMMIT_GRAPH_WRITE_REACHABLE) > 1)\n \t\tBUG(\"attempting to write commit-graph with mutually incompatible options\");\n \n-\tprepare_repo_settings(r);\n-\tif (!r->settings.core_commit_graph) {\n-\t\twarning(_(\"attempting to write a commit-graph, but 'core.commitGraph' is disabled\"));\n+\tif (!commit_graph_write_allowed(r))\n \t\treturn 0;\n-\t}\n-\tif (!commit_graph_compatible(r))\n-\t\treturn 0;\n-\tif (r->settings.commit_graph_changed_paths_version < -1\n-\t    || r->settings.commit_graph_changed_paths_version > 2) {\n-\t\twarning(_(\"attempting to write a commit-graph, but \"\n-\t\t\t  \"'commitGraph.changedPathsVersion' (%d) is not supported\"),\n-\t\t\tr->settings.commit_graph_changed_paths_version);\n-\t\treturn 0;\n-\t}\n \n \tctx.dir = absolute_pathdup(dir);\n \ndiff --git a/commit-graph.h b/commit-graph.h\nindex dc4f6194c72..e2fc0c8da4c 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -198,6 +198,9 @@ struct commit_graph_write_options {\n \t.max_new_filters = -1, \\\n }\n \n+/* Check whether the repository's configuration allows writing commit graphs. */\n+bool commit_graph_write_allowed(struct repository *r);\n+\n /*\n  * The commit_graph_write* methods return zero on success\n  * and a negative value on failure. Note that if the repository\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558543","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-8-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 08/12] commit-graph: split out logic to write a new commit graph layer","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:44Z","receivedAt":"2026-11-16T13:07:44Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The logic to write a new commit graph is mostly contained in\n`write_commit_graph_file()`. This function has two different\nresponsibilities:\n\n  - It writes the content of the new commit graph layer.\n\n  - It manages the location of the newly written files, as well as the\n    deletion of old files.\n\nWhile the former is generic functionality, the latter is specific to the\nactual backend once we make commit graphs pluggable.\n\nSeparate these two concerns by introducing `commit_graph_write_layer()`,\nwhich is only responsible for writing the content of the new file.\nThe backend-specific handling of paths remain the responsibility of\n`commit_graph_write()`, which allows us to move that logic into the\nbackend in a subsequent commit.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n commit-graph.c | 247 ++++++++++++++++++++++++++++++++-------------------------\n commit-graph.h |  27 +++++++\n 2 files changed, 166 insertions(+), 108 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex c673c0424da..7bb6b112b2e 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -1140,8 +1140,6 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit\n \n struct write_commit_graph_context {\n \tstruct repository *r;\n-\tchar *dir;\n-\tchar *graph_name;\n \tstruct oid_array oids;\n \tstruct commit_stack commits;\n \tint num_extra_edges;\n@@ -2092,68 +2090,16 @@ static void expire_commit_graphs(const char *dirpath,\n \t\t\t\t struct string_list *kept,\n \t\t\t\t timestamp_t expire_time);\n \n-static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n+static int write_commit_graph_file(struct write_commit_graph_context *ctx,\n+\t\t\t\t   int fd, const char *path,\n+\t\t\t\t   unsigned char *file_hash)\n {\n-\tstruct hashfile *f;\n-\tstruct tempfile *graph_layer = NULL; /* when COMMIT_GRAPH_WRITE_SPLIT is non-zero */\n-\tstruct lock_file lk = LOCK_INIT;\n \tconst unsigned hashsz = ctx->r->hash_algo->rawsz;\n \tstruct strbuf progress_title = STRBUF_INIT;\n-\tstruct string_list chain_hashes = STRING_LIST_INIT_DUP;\n-\tstruct string_list kept = STRING_LIST_INIT_DUP;\n-\tstruct commit_graph *base = ctx->new_base_graph, *chain;\n-\tchar *base_rename_from = NULL, *base_rename_to = NULL;\n-\tchar *new_path = NULL;\n+\tstruct hashfile *f;\n \tstruct chunkfile *cf;\n-\tunsigned char file_hash[GIT_MAX_RAWSZ];\n-\tint ret;\n-\n-\tif (ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tstruct strbuf tmp_file = STRBUF_INIT;\n-\n-\t\tstrbuf_addf(&tmp_file,\n-\t\t\t    \"%s/info/commit-graphs/tmp_graph_XXXXXX\",\n-\t\t\t    ctx->dir);\n-\t\tctx->graph_name = strbuf_detach(&tmp_file, NULL);\n-\t} else {\n-\t\tctx->graph_name = get_commit_graph_filename(ctx->dir);\n-\t}\n-\n-\tif (safe_create_leading_directories(ctx->r, ctx->graph_name)) {\n-\t\tret = error(_(\"unable to create leading directories of %s\"),\n-\t\t\t    ctx->graph_name);\n-\t\tgoto out;\n-\t}\n-\n-\tif (ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tchar *lock_name = get_commit_graph_chain_filename(ctx->dir);\n-\n-\t\trepo_hold_lock_file_for_update_mode(ctx->r, &lk, lock_name,\n-\t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n-\t\tfree(lock_name);\n-\n-\t\tgraph_layer = mks_tempfile_m(ctx->graph_name, 0444);\n-\t\tif (!graph_layer) {\n-\t\t\tret = error(_(\"unable to create temporary graph layer\"));\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\tif (adjust_shared_perm(ctx->r, get_tempfile_path(graph_layer))) {\n-\t\t\tret = error(_(\"unable to adjust shared permissions for '%s'\"),\n-\t\t\t\t    get_tempfile_path(graph_layer));\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\tf = hashfd(ctx->r->hash_algo,\n-\t\t\t   get_tempfile_fd(graph_layer), get_tempfile_path(graph_layer));\n-\t} else {\n-\t\trepo_hold_lock_file_for_update_mode(ctx->r, &lk,\n-\t\t\t\t\t\t    ctx->graph_name,\n-\t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n-\t\tf = hashfd(ctx->r->hash_algo,\n-\t\t\t   get_lock_file_fd(&lk), get_lock_file_path(&lk));\n-\t}\n \n+\tf = hashfd(ctx->r->hash_algo, fd, path);\n \tcf = init_chunkfile(f);\n \n \tadd_chunk(cf, GRAPH_CHUNKID_OIDFANOUT, GRAPH_FANOUT_SIZE,\n@@ -2217,14 +2163,114 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \t\t\t  CSUM_HASH_IN_STREAM | CSUM_FSYNC);\n \tfree_chunkfile(cf);\n \n+\treturn 0;\n+}\n+\n+int commit_graph_write(struct repository *r,\n+\t\t       const char *dir,\n+\t\t       const struct commit_graph_write_options *opts)\n+{\n+\tstruct string_list chain_hashes = STRING_LIST_INIT_DUP;\n+\tstruct string_list kept = STRING_LIST_INIT_DUP;\n+\tstruct tempfile *graph_layer = NULL;\n+\tstruct lock_file lk = LOCK_INIT;\n+\tstruct commit_graph *g, *chain, *base = NULL;\n+\tchar *base_rename_from = NULL, *base_rename_to = NULL;\n+\tchar *graph_name = NULL, *abs_dir = NULL, *new_path = NULL;\n+\tsize_t mergeable_layers = 0, chainable_layers = 0;\n+\tunsigned char hash[GIT_MAX_RAWSZ];\n+\tbool mergeable = true;\n+\tconst char *out_path;\n+\tint out_fd;\n+\tint ret;\n+\n+\tif (!commit_graph_write_allowed(r))\n+\t\treturn 0;\n+\n+\tabs_dir = absolute_pathdup(dir);\n+\n+\tg = prepare_commit_graph(r);\n+\n+\t/*\n+\t * We only consider merging layers that belong to our own object\n+\t * directory. Furthermore, we can only chain on top of either:\n+\t *\n+\t *   - A split commit graph layer, regardless of whether or not it is\n+\t *     local.\n+\t *\n+\t *   - A local non-split commit graph, in which case we'll convert that\n+\t *     layer into a split commit graph layer.\n+\t */\n+\tfor (struct commit_graph *layer = g; layer; layer = layer->base_graph) {\n+\t\tchar *path = get_commit_graph_filename(layer->dir);\n+\t\tbool is_split = !!strcmp(layer->filename, path);\n+\t\tbool is_local = !strcmp(layer->dir, abs_dir);\n+\n+\t\tfree(path);\n+\n+\t\tif (!is_local && !is_split)\n+\t\t\tbreak;\n+\t\tif (!is_local)\n+\t\t\tmergeable = false;\n+\n+\t\tif (mergeable)\n+\t\t\tmergeable_layers++;\n+\t\tchainable_layers++;\n+\t}\n+\n+\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\t\tgraph_name = xstrfmt(\"%s/info/commit-graphs/tmp_graph_XXXXXX\", abs_dir);\n+\t} else {\n+\t\tgraph_name = get_commit_graph_filename(abs_dir);\n+\t}\n+\n+\tif (safe_create_leading_directories(r, graph_name)) {\n+\t\tret = error(_(\"unable to create leading directories of %s\"),\n+\t\t\t    graph_name);\n+\t\tgoto out;\n+\t}\n+\n+\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\t\tchar *chain_name = get_commit_graph_chain_filename(abs_dir);\n+\n+\t\trepo_hold_lock_file_for_update_mode(r, &lk, chain_name,\n+\t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n+\t\tfree(chain_name);\n+\n+\t\tgraph_layer = mks_tempfile_m(graph_name, 0444);\n+\t\tif (!graph_layer) {\n+\t\t\tret = error(_(\"unable to create temporary graph layer\"));\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (adjust_shared_perm(r, get_tempfile_path(graph_layer))) {\n+\t\t\tret = error(_(\"unable to adjust shared permissions for '%s'\"),\n+\t\t\t\t    get_tempfile_path(graph_layer));\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tout_fd = get_tempfile_fd(graph_layer);\n+\t\tout_path = get_tempfile_path(graph_layer);\n+\t} else {\n+\t\trepo_hold_lock_file_for_update_mode(r, &lk, graph_name,\n+\t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n+\t\tout_fd = get_lock_file_fd(&lk);\n+\t\tout_path = get_lock_file_path(&lk);\n+\t}\n+\n+\tret = commit_graph_write_layer(r, opts, mergeable_layers, chainable_layers,\n+\t\t\t\t       out_fd, out_path, &base, hash);\n+\tif (ret <= 0)\n+\t\tgoto out;\n+\n \t/*\n \t * Snapshot everything we need from the current chain before closing\n \t * it, as we must not have any layers mapped when replacing them.\n \t */\n-\tif (ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tconst char *new_hash = hash_to_hex_algop(file_hash, ctx->r->hash_algo);\n+\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\t\tconst char *new_hash = hash_to_hex_algop(hash, r->hash_algo);\n \n-\t\tnew_path = get_split_graph_filename(ctx->dir, new_hash);\n+\t\tnew_path = get_split_graph_filename(abs_dir, new_hash);\n \n \t\tstring_list_append(&chain_hashes, new_hash);\n \t\tfor (chain = base; chain; chain = chain->base_graph)\n@@ -2251,12 +2297,12 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \t\t}\n \t\tstring_list_append(&kept, new_path);\n \n-\t\tmark_commit_graphs(ctx->r->objects->commit_graph, base);\n+\t\tmark_commit_graphs(g, base);\n \t}\n \n-\tcommit_graph_close(ctx->r->objects);\n+\tcommit_graph_close(r->objects);\n \n-\tif (ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n \t\tFILE *chainf = fdopen_lock_file(&lk, \"w\");\n \n \t\tif (!chainf) {\n@@ -2270,7 +2316,7 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \t\t\t\tgoto out;\n \t\t\t}\n \t\t} else if (!base) {\n-\t\t\tchar *graph_name = get_commit_graph_filename(ctx->dir);\n+\t\t\tchar *graph_name = get_commit_graph_filename(abs_dir);\n \t\t\tunlink(graph_name);\n \t\t\tfree(graph_name);\n \t\t}\n@@ -2290,14 +2336,14 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \n \tcommit_lock_file(&lk);\n \n-\tif (!(ctx->opts->flags & COMMIT_GRAPH_WRITE_SPLIT)) {\n-\t\tchar *chain_name = get_commit_graph_chain_filename(ctx->dir);\n+\tif (!(opts->flags & COMMIT_GRAPH_WRITE_SPLIT)) {\n+\t\tchar *chain_name = get_commit_graph_chain_filename(abs_dir);\n \t\tunlink(chain_name);\n \t\tfree(chain_name);\n \t}\n \n-\texpire_commit_graphs(ctx->dir, &kept,\n-\t\t\t     ctx->opts->expire_time ? ctx->opts->expire_time : time(NULL));\n+\texpire_commit_graphs(abs_dir, &kept,\n+\t\t\t     opts->expire_time ? opts->expire_time : time(NULL));\n \n \tret = 0;\n \n@@ -2308,7 +2354,9 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx)\n \tstring_list_clear(&kept, 0);\n \tfree(base_rename_from);\n \tfree(base_rename_to);\n+\tfree(graph_name);\n \tfree(new_path);\n+\tfree(abs_dir);\n \treturn ret;\n }\n \n@@ -2562,13 +2610,19 @@ bool commit_graph_write_allowed(struct repository *r)\n \treturn true;\n }\n \n-int commit_graph_write(struct repository *r,\n-\t\t       const char *dir,\n-\t\t       const struct commit_graph_write_options *opts)\n+int commit_graph_write_layer(struct repository *r,\n+\t\t\t     const struct commit_graph_write_options *opts,\n+\t\t\t     size_t mergeable_layers,\n+\t\t\t     size_t chainable_layers,\n+\t\t\t     int fd, const char *path,\n+\t\t\t     struct commit_graph **base,\n+\t\t\t     unsigned char *hash)\n {\n \tstruct write_commit_graph_context ctx = {\n \t\t.r = r,\n \t\t.opts = opts,\n+\t\t.mergeable_layers = mergeable_layers,\n+\t\t.chainable_layers = chainable_layers,\n \t\t.total_bloom_filter_data_size = 0,\n \t\t.write_generation_data = (get_configured_generation_version(r) == 2),\n \t\t.num_generation_data_overflows = 0,\n@@ -2580,16 +2634,17 @@ int commit_graph_write(struct repository *r,\n \tstruct oidset reachable_commits = OIDSET_INIT;\n \tstruct topo_level_slab topo_levels;\n \tstruct commit_graph *g;\n-\tbool mergeable = true;\n \n \tif (!!opts->pack_indexes + !!opts->commits + !!(opts->flags & COMMIT_GRAPH_WRITE_REACHABLE) > 1)\n \t\tBUG(\"attempting to write commit-graph with mutually incompatible options\");\n+\tif (chainable_layers < mergeable_layers)\n+\t\tBUG(\"attempting to write commit-graph with more mergeable than chainable layers\");\n+\n+\t*base = NULL;\n \n \tif (!commit_graph_write_allowed(r))\n \t\treturn 0;\n \n-\tctx.dir = absolute_pathdup(dir);\n-\n \tbloom_settings.hash_version = r->settings.commit_graph_changed_paths_version;\n \tbloom_settings.bits_per_entry = git_env_ulong(\"GIT_TEST_BLOOM_SETTINGS_BITS_PER_ENTRY\",\n \t\t\t\t\t\t      bloom_settings.bits_per_entry);\n@@ -2606,33 +2661,6 @@ int commit_graph_write(struct repository *r,\n \tfor (struct commit_graph *chain = g; chain; chain = chain->base_graph)\n \t\tchain->topo_levels = &topo_levels;\n \n-\t/*\n-\t * We only consider merging layers that belong to our own object\n-\t * directory. Furthermore, we can only chain on top of either:\n-\t *\n-\t *   - A split commit graph layer, regardless of whether or not it is\n-\t *     local.\n-\t *\n-\t *   - A local non-split commit graph, in which case we'll convert that\n-\t *     layer into a split commit graph layer.\n-\t */\n-\tfor (struct commit_graph *layer = g; layer; layer = layer->base_graph) {\n-\t\tchar *path = get_commit_graph_filename(layer->dir);\n-\t\tbool is_split = !!strcmp(layer->filename, path);\n-\t\tbool is_local = !strcmp(layer->dir, ctx.dir);\n-\n-\t\tfree(path);\n-\n-\t\tif (!is_local && !is_split)\n-\t\t\tbreak;\n-\t\tif (!is_local)\n-\t\t\tmergeable = false;\n-\n-\t\tif (mergeable)\n-\t\t\tctx.mergeable_layers++;\n-\t\tctx.chainable_layers++;\n-\t}\n-\n \tif (opts->flags & COMMIT_GRAPH_WRITE_BLOOM_FILTERS)\n \t\tctx.changed_paths = 1;\n \tif (!(opts->flags & COMMIT_GRAPH_NO_WRITE_BLOOM_FILTERS)) {\n@@ -2717,21 +2745,24 @@ int commit_graph_write(struct repository *r,\n \tif (ctx.changed_paths)\n \t\tcompute_bloom_filters(&ctx);\n \n-\tres = write_commit_graph_file(&ctx);\n+\tres = write_commit_graph_file(&ctx, fd, path, hash);\n \n \tif (ctx.changed_paths)\n \t\tdeinit_bloom_filters();\n \n+\tif (!res) {\n+\t\t*base = ctx.new_base_graph;\n+\t\tres = 1;\n+\t}\n+\n cleanup:\n-\tfree(ctx.graph_name);\n-\tfree(ctx.dir);\n \tcommit_stack_clear(&ctx.commits);\n \toidset_clear(&reachable_commits);\n \toid_array_clear(&ctx.oids);\n \tclear_topo_level_slab(&topo_levels);\n \n-\tif (ctx.r->objects->commit_graph) {\n-\t\tstruct commit_graph *g = ctx.r->objects->commit_graph;\n+\tif (r->objects->commit_graph) {\n+\t\tstruct commit_graph *g = r->objects->commit_graph;\n \n \t\twhile (g) {\n \t\t\tg->topo_levels = NULL;\ndiff --git a/commit-graph.h b/commit-graph.h\nindex e2fc0c8da4c..1cc0d097321 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -201,6 +201,33 @@ struct commit_graph_write_options {\n /* Check whether the repository's configuration allows writing commit graphs. */\n bool commit_graph_write_allowed(struct repository *r);\n \n+/*\n+ * Write a single commit-graph layer to the given file descriptor. When writing\n+ * a split commit graph then the repository's loaded commit graph will be used\n+ * as the base. This can be tweaked:\n+ *\n+ *   - `mergeable_layers` controls how many of the topmost layers may be merged\n+ *     together.\n+ *\n+ *   - `chainable_layers` controls how many of the topmost layers may be used\n+ *     as base. In case the picked base is not chainable, then it and all of\n+ *     its bases are merged.\n+ *\n+ * On success, `*base` is set to the layer of the currently-loaded chain that\n+ * the new layer builds upon, or NULL if the new layer is standalone. `hash`\n+ * receives the trailing checksum of the new layer.\n+ *\n+ * Returns 1 in case a new layer has been written, 0 in case there was nothing\n+ * to write, a negative error code otherwise.\n+ */\n+int commit_graph_write_layer(struct repository *r,\n+\t\t\t     const struct commit_graph_write_options *opts,\n+\t\t\t     size_t mergeable_layers,\n+\t\t\t     size_t chainable_layers,\n+\t\t\t     int fd, const char *path,\n+\t\t\t     struct commit_graph **base,\n+\t\t\t     unsigned char *hash);\n+\n /*\n  * The commit_graph_write* methods return zero on success\n  * and a negative value on failure. Note that if the repository\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558544","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-9-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 09/12] commit-graph: mark incomplete commit graph chains","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:45Z","receivedAt":"2026-11-16T13:07:45Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"When loading a commit graph chain it may happen that we only find a\nsubset of its layers. Such a commit graph is still usable though in the\ncase where we have the base and layers building on top of it, so we\nstill return such an incomplete commit graph to the caller.\n\nWhile valid and usable, we of course want to warn about such commit\ngraphs in `git commit-graph verify`, which is why there is an\n`incomplete` out-parameter in `load_commit_graph_chain_fd_st()`. In a\nsubsequent commit though we want to hide this function in favor of a\npluggable interface, but carrying such a flag there feels wrong.\n\nInstead, have the function set a new `incomplete` field in the commit\ngraph structure itself.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/commit-graph.c | 9 ++++-----\n commit-graph.c         | 9 ++++-----\n commit-graph.h         | 4 ++--\n 3 files changed, 10 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\nindex 2a5a75dc22f..5ecce4b6caa 100644\n--- a/builtin/commit-graph.c\n+++ b/builtin/commit-graph.c\n@@ -66,7 +66,6 @@ static int cmd_commit_graph_verify(int argc,\n \tenum commit_graph_verify_flags flags = isatty(2) ? COMMIT_GRAPH_VERIFY_PROGRESS : 0;\n \tint fd;\n \tstruct stat st;\n-\tint incomplete_chain = 0;\n \tint ret;\n \tstruct option options[] = {\n \t\tOPT_BIT(0, \"shallow\", &flags,\n@@ -113,20 +112,20 @@ static int cmd_commit_graph_verify(int argc,\n \telse if (opened == OPENED_GRAPH)\n \t\tgraph = commit_graph_read(repo, fd, &st);\n \telse\n-\t\tgraph = load_commit_graph_chain_fd_st(repo->objects, fd, &st,\n-\t\t\t\t\t\t      &incomplete_chain);\n+\t\tgraph = load_commit_graph_chain_fd_st(repo->objects, fd, &st);\n \n \tif (!graph)\n \t\treturn 1;\n \n \tret = commit_graph_verify(graph, flags);\n-\tcommit_graph_free(graph);\n \n-\tif (incomplete_chain) {\n+\tif (graph->incomplete) {\n \t\terror(\"one or more commit-graph chain files could not be loaded\");\n \t\tret |= 1;\n \t}\n \n+\tcommit_graph_free(graph);\n+\n \treturn ret;\n }\n \ndiff --git a/commit-graph.c b/commit-graph.c\nindex 7bb6b112b2e..0ae9f586f21 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -651,8 +651,7 @@ int open_commit_graph_chain(const char *chain_file,\n }\n \n struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n-\t\t\t\t\t\t   int fd, struct stat *st,\n-\t\t\t\t\t\t   int *incomplete_chain)\n+\t\t\t\t\t\t   int fd, struct stat *st)\n {\n \tstruct commit_graph *graph_chain = NULL;\n \tstruct strbuf line = STRBUF_INIT;\n@@ -703,11 +702,12 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n \t}\n \n \tcommit_graph_validate_chain(graph_chain);\n+\tif (graph_chain)\n+\t\tgraph_chain->incomplete = !valid;\n \n \tfclose(fp);\n \tstrbuf_release(&line);\n \n-\t*incomplete_chain = !valid;\n \treturn graph_chain;\n }\n \n@@ -720,9 +720,8 @@ static struct commit_graph *load_commit_graph_chain(struct repository *repo,\n \tstruct commit_graph *g = NULL;\n \n \tif (open_commit_graph_chain(chain_file, &fd, &st, repo->hash_algo)) {\n-\t\tint incomplete;\n \t\t/* ownership of fd is taken over by load function */\n-\t\tg = load_commit_graph_chain_fd_st(repo->objects, fd, &st, &incomplete);\n+\t\tg = load_commit_graph_chain_fd_st(repo->objects, fd, &st);\n \t}\n \n \tfree(chain_file);\ndiff --git a/commit-graph.h b/commit-graph.h\nindex 1cc0d097321..3b5e0eb2d73 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -93,6 +93,7 @@ struct commit_graph {\n \tstruct object_id oid;\n \tchar *dir;\n \tchar *filename;\n+\tbool incomplete;\n \n \tuint32_t num_commits_in_base;\n \tunsigned int read_generation_data;\n@@ -124,8 +125,7 @@ struct commit_graph {\n struct commit_graph *commit_graph_read(struct repository *repo,\n \t\t\t\t       int fd, struct stat *st);\n struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n-\t\t\t\t\t\t   int fd, struct stat *st,\n-\t\t\t\t\t\t   int *incomplete_chain);\n+\t\t\t\t\t\t   int fd, struct stat *st);\n \n /*\n  * Link the commit graph layer `g` on top of the chain `base`. `base` may be\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558545","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-10-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 10/12] odb/source: make commit graphs pluggable","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:46Z","receivedAt":"2026-11-16T13:07:46Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"The location where commit graphs are read from and written to is\nhardcoded to \".git/objects/info\". This indeed makes sense for the\n\"files\" backend, which also stores all of its other information in these\ndirectories. But other backends may want to store commit graphs\ndifferently, if they even want to support them at all.\n\nGive backends control over how commit graphs are stored by making the\nability to both read and write commit graphs pluggable. Adapt callers to\nuse the new, backend-generic functions where possible. The one exception\nis \"builtin/commit-graph.c\", whose \"--object-dir=\" option allows the\nuser to specify the target object directory. This is highly specific to\nthe \"files\" backend though, and consequently we only support that option\nthere.\n\nNote that the logic that locates commit graphs is still hosted by\n\"commit-graph.c\" right now. This will be fixed in a subsequent commit.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/commit-graph.c     | 25 +++++++++++++++-------\n builtin/fetch.c            |  4 +---\n builtin/gc.c               |  3 +--\n commit-graph.c             | 13 ++++--------\n odb.c                      |  6 ++++++\n odb.h                      | 10 +++++++++\n odb/source-files.c         | 27 ++++++++++++++++++++++++\n odb/source.h               | 52 ++++++++++++++++++++++++++++++++++++++++++++++\n t/helper/test-read-graph.c |  3 +--\n 9 files changed, 119 insertions(+), 24 deletions(-)\n\ndiff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\nindex 5ecce4b6caa..6ee644d8487 100644\n--- a/builtin/commit-graph.c\n+++ b/builtin/commit-graph.c\n@@ -277,23 +277,27 @@ static int cmd_commit_graph_write(int argc,\n \n \tif (!!(write_opts.flags & COMMIT_GRAPH_WRITE_REACHABLE) + opts.stdin_packs + opts.stdin_commits > 1)\n \t\tdie(_(\"use at most one of --reachable, --stdin-commits, or --stdin-packs\"));\n-\tif (!opts.obj_dir)\n-\t\topts.obj_dir = repo_get_object_directory(repo);\n \tif (!opts.enable_changed_paths)\n \t\twrite_opts.flags |= COMMIT_GRAPH_NO_WRITE_BLOOM_FILTERS;\n \tif (opts.enable_changed_paths == 1 ||\n \t    git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))\n \t\twrite_opts.flags |= COMMIT_GRAPH_WRITE_BLOOM_FILTERS;\n \n-\tdir = odb_source_files_find_dir(repo->objects, opts.obj_dir);\n-\tif (!dir)\n-\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n+\tif (opts.obj_dir) {\n+\t\tdir = odb_source_files_find_dir(repo->objects, opts.obj_dir);\n+\t\tif (!dir)\n+\t\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n+\t}\n \n \tif (opts.stdin_packs) {\n \t\tstruct strbuf packname = STRBUF_INIT;\n \t\tsize_t dirlen;\n \n-\t\tstrbuf_addf(&packname, \"%s/pack/\", dir->abspath);\n+\t\tif (dir)\n+\t\t\tstrbuf_addf(&packname, \"%s/pack/\", dir->abspath);\n+\t\telse\n+\t\t\tstrbuf_addf(&packname, \"%s/pack/\",\n+\t\t\t\t    repo_get_object_directory(repo));\n \t\tdirlen = packname.len;\n \n \t\twhile (strbuf_getline(&buf, stdin) != EOF) {\n@@ -323,8 +327,13 @@ static int cmd_commit_graph_write(int argc,\n \t\tstop_progress(&progress);\n \t}\n \n-\tif (commit_graph_write(repo, dir->abspath, &write_opts))\n-\t\tresult = 1;\n+\tif (dir) {\n+\t\tif (commit_graph_write(repo, dir->abspath, &write_opts))\n+\t\t\tresult = 1;\n+\t} else {\n+\t\tif (odb_write_commit_graph(repo->objects, &write_opts))\n+\t\t\tresult = 1;\n+\t}\n \n cleanup:\n \tstring_list_clear(&pack_indexes, 0);\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex c646bc6698d..b7606016817 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -2978,9 +2978,7 @@ int cmd_fetch(int argc,\n \t\tif (graph_write_mode != GRAPH_WRITE_SKIP) {\n \t\t\ttrace2_region_enter(\"fetch\", \"write-commit-graph\",\n \t\t\t\t\t    the_repository);\n-\t\t\tcommit_graph_write(the_repository,\n-\t\t\t\t\t   the_repository->objects->source->path,\n-\t\t\t\t\t   &write_opts);\n+\t\t\todb_write_commit_graph(the_repository->objects, &write_opts);\n \t\t\ttrace2_region_leave(\"fetch\", \"write-commit-graph\",\n \t\t\t\t\t    the_repository);\n \t\t}\ndiff --git a/builtin/gc.c b/builtin/gc.c\nindex 68cb04a672d..f9a15912d0e 100644\n--- a/builtin/gc.c\n+++ b/builtin/gc.c\n@@ -741,8 +741,7 @@ int cmd_gc(int argc,\n \t\tif (!opts.quiet && !daemonized)\n \t\t\twrite_opts.flags |= COMMIT_GRAPH_WRITE_PROGRESS;\n \n-\t\tcommit_graph_write(the_repository, the_repository->objects->source->path,\n-\t\t\t\t   &write_opts);\n+\t\todb_write_commit_graph(the_repository->objects, &write_opts);\n \t}\n \n \tif (!daemonized) {\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 0ae9f586f21..7fea94689a1 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -42,7 +42,7 @@ void git_test_write_commit_graph_or_die(struct repository *repo)\n \tif (git_env_bool(GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS, 0))\n \t\topts.flags |= COMMIT_GRAPH_WRITE_BLOOM_FILTERS;\n \n-\tif (commit_graph_write(repo, repo->objects->source->path, &opts))\n+\tif (odb_write_commit_graph(repo->objects, &opts))\n \t\tdie(\"failed to write commit-graph under GIT_TEST_COMMIT_GRAPH\");\n }\n \n@@ -747,8 +747,6 @@ struct commit_graph *read_commit_graph_one(struct repository *repo,\n  */\n struct commit_graph *prepare_commit_graph(struct repository *r)\n {\n-\tstruct odb_source_files *files;\n-\n \t/*\n \t * Early return if there is no object database or if the commit graph is\n \t * disabled.\n@@ -778,12 +776,9 @@ struct commit_graph *prepare_commit_graph(struct repository *r)\n \tif (!commit_graph_compatible(r))\n \t\treturn NULL;\n \n-\tfiles = odb_source_files_downcast(r->objects->source);\n-\tfor (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {\n-\t\tr->objects->commit_graph = read_commit_graph_one(r, dir->abspath);\n-\t\tif (r->objects->commit_graph)\n-\t\t\tbreak;\n-\t}\n+\tif (odb_source_read_commit_graph(r->objects->source,\n+\t\t\t\t\t &r->objects->commit_graph))\n+\t\treturn NULL;\n \n \treturn r->objects->commit_graph;\n }\ndiff --git a/odb.c b/odb.c\nindex 5dd277761b1..ec5064c3cd0 100644\n--- a/odb.c\n+++ b/odb.c\n@@ -839,3 +839,9 @@ int odb_fsck(struct object_database *odb, struct odb_fsck_options *options)\n {\n \treturn odb_source_fsck(odb->source, options);\n }\n+\n+int odb_write_commit_graph(struct object_database *odb,\n+\t\t\t   const struct commit_graph_write_options *opts)\n+{\n+\treturn odb_source_write_commit_graph(odb->source, opts);\n+}\ndiff --git a/odb.h b/odb.h\nindex 857637f9805..099ba998faa 100644\n--- a/odb.h\n+++ b/odb.h\n@@ -221,6 +221,16 @@ struct odb_fsck_options {\n  */\n int odb_fsck(struct object_database *odb, struct odb_fsck_options *opts);\n \n+struct commit_graph_write_options;\n+\n+/*\n+ * Write a new commit graph layer into the primary object source. The source\n+ * is in control of where and how the layer is stored. Returns 0 on success, a\n+ * negative error code otherwise.\n+ */\n+int odb_write_commit_graph(struct object_database *odb,\n+\t\t\t   const struct commit_graph_write_options *opts);\n+\n /*\n  * Iterate through all alternates of the database and execute the provided\n  * callback function for each of them. Stop iterating once the callback\ndiff --git a/odb/source-files.c b/odb/source-files.c\nindex 0bf15031c87..5ad27fd4bc7 100644\n--- a/odb/source-files.c\n+++ b/odb/source-files.c\n@@ -2,6 +2,7 @@\n #include \"abspath.h\"\n #include \"blob.h\"\n #include \"chdir-notify.h\"\n+#include \"commit-graph.h\"\n #include \"config.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n@@ -1230,6 +1231,30 @@ static int odb_source_files_generate_pack(struct odb_source *source UNUSED,\n \treturn 0;\n }\n \n+static int odb_source_files_read_commit_graph(struct odb_source *source,\n+\t\t\t\t\t      struct commit_graph **out)\n+{\n+\tstruct odb_source_files *files = odb_source_files_downcast(source);\n+\n+\tfor (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {\n+\t\tstruct commit_graph *g = read_commit_graph_one(files->base.odb->repo,\n+\t\t\t\t\t\t\t       dir->abspath);\n+\t\tif (!g)\n+\t\t\tcontinue;\n+\t\t*out = g;\n+\t\treturn 0;\n+\t}\n+\n+\treturn 1;\n+}\n+\n+static int odb_source_files_write_commit_graph(struct odb_source *source,\n+\t\t\t\t\t       const struct commit_graph_write_options *opts)\n+{\n+\tstruct odb_source_files *files = odb_source_files_downcast(source);\n+\treturn commit_graph_write(source->odb->repo, files->dirs->abspath, opts);\n+}\n+\n static int fsck_commit_graph(struct odb_source_files *files,\n \t\t\t     struct odb_files_dir *dir,\n \t\t\t     struct odb_fsck_options *opts)\n@@ -1335,6 +1360,8 @@ struct odb_source_files *odb_source_files_new(struct object_database *odb,\n \tfiles->base.optimize = odb_source_files_optimize;\n \tfiles->base.optimize_required = odb_source_files_optimize_required;\n \tfiles->base.generate_pack = odb_source_files_generate_pack;\n+\tfiles->base.read_commit_graph = odb_source_files_read_commit_graph;\n+\tfiles->base.write_commit_graph = odb_source_files_write_commit_graph;\n \n \t/*\n \t * Ideally, we would only ever store absolute paths in the source. This\ndiff --git a/odb/source.h b/odb/source.h\nindex 7b56f51b827..bc9604f7df9 100644\n--- a/odb/source.h\n+++ b/odb/source.h\n@@ -1,6 +1,7 @@\n #ifndef ODB_SOURCE_H\n #define ODB_SOURCE_H\n \n+#include \"gettext.h\"\n #include \"hashmap.h\"\n #include \"object.h\"\n #include \"odb.h\"\n@@ -32,6 +33,8 @@ enum odb_source_type {\n  */\n const char *odb_source_type_to_name(enum odb_source_type type);\n \n+struct commit_graph;\n+struct commit_graph_write_options;\n struct object_id;\n struct odb_stream;\n struct strbuf;\n@@ -285,6 +288,29 @@ struct odb_source {\n \t */\n \tint (*fsck)(struct odb_source *source,\n \t\t    struct odb_fsck_options *options);\n+\n+\t/*\n+\t * This callback is expected to read the commit graph of the source,\n+\t * if any. This callback is optional. Sources that do not support\n+\t * commit graphs shall leave it unset.\n+\t *\n+\t * The callback is expected to return 0 on success, 1 in case no commit\n+\t * graph exists and a negative error code otherwise. On success, `out`\n+\t * must be set to the commit graph.\n+\t */\n+\tint (*read_commit_graph)(struct odb_source *source,\n+\t\t\t\t struct commit_graph **out);\n+\n+\t/*\n+\t * This callback is expected to write a new commit graph layer with\n+\t * the given options. This callback is optional. Sources that do not\n+\t * support commit graphs shall leave it unset.\n+\t *\n+\t * The callback is expected to return 0 on success, a negative error\n+\t * code otherwise.\n+\t */\n+\tint (*write_commit_graph)(struct odb_source *source,\n+\t\t\t\t  const struct commit_graph_write_options *opts);\n };\n \n /*\n@@ -540,4 +566,30 @@ static inline int odb_source_fsck(struct odb_source *source,\n \treturn source->fsck(source, opts);\n }\n \n+/*\n+ * Read the commit graph of the given source. Returns 0 on success, 1 in case\n+ * no commit graph exists, and a negative error code otherwise.\n+ */\n+static inline int odb_source_read_commit_graph(struct odb_source *source,\n+\t\t\t\t\t       struct commit_graph **out)\n+{\n+\tif (!source->read_commit_graph)\n+\t\treturn 1;\n+\treturn source->read_commit_graph(source, out);\n+}\n+\n+/*\n+ * Write a new commit graph layer into the given source. Returns 0 on success,\n+ * a negative error code otherwise. Sources that do not support commit graphs\n+ * cause an error.\n+ */\n+static inline int odb_source_write_commit_graph(struct odb_source *source,\n+\t\t\t\t\t\tconst struct commit_graph_write_options *opts)\n+{\n+\tif (!source->write_commit_graph)\n+\t\treturn error(_(\"object source '%s' does not support commit graphs\"),\n+\t\t\t     odb_source_type_to_name(source->type));\n+\treturn source->write_commit_graph(source, opts);\n+}\n+\n #endif\ndiff --git a/t/helper/test-read-graph.c b/t/helper/test-read-graph.c\nindex 9642e55b8c0..d6174bb55f4 100644\n--- a/t/helper/test-read-graph.c\n+++ b/t/helper/test-read-graph.c\n@@ -85,8 +85,7 @@ int cmd__read_graph(int argc, const char **argv)\n \n \tprepare_repo_settings(the_repository);\n \n-\tgraph = read_commit_graph_one(the_repository, source->path);\n-\tif (!graph) {\n+\tif (odb_source_read_commit_graph(source, &graph)) {\n \t\tret = 1;\n \t\tgoto done;\n \t}\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558546","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-11-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 11/12] odb/source-files: move logic to derive commit graph paths","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:47Z","receivedAt":"2026-11-16T13:07:47Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"Move the logic that derives commit graph paths from the generic\ncommit-graph subsystem into the \"files\" backend.\n\nThis is mostly an exact move, and we still carry declarations in\n\"commit-graph.h\", so that it becomes trivial to see that nothing else\nhas changed. The only exception are three sites where we now generate\nwarnings because of -Wsign-compare.\n\nThe next commit will refactor moved code so that we can drop those\ndeclarations.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n commit-graph.c     | 437 -----------------------------------------------------\n commit-graph.h     |   2 -\n odb/source-files.c | 428 +++++++++++++++++++++++++++++++++++++++++++++++++++\n 3 files changed, 428 insertions(+), 439 deletions(-)\n\ndiff --git a/commit-graph.c b/commit-graph.c\nindex 7fea94689a1..8daadf56e88 100644\n--- a/commit-graph.c\n+++ b/commit-graph.c\n@@ -1,13 +1,11 @@\n #define DISABLE_SIGN_COMPARE_WARNINGS\n \n #include \"git-compat-util.h\"\n-#include \"abspath.h\"\n #include \"config.h\"\n #include \"csum-file.h\"\n #include \"environment.h\"\n #include \"gettext.h\"\n #include \"hex.h\"\n-#include \"lockfile.h\"\n #include \"packfile.h\"\n #include \"commit.h\"\n #include \"object.h\"\n@@ -25,7 +23,6 @@\n #include \"bloom.h\"\n #include \"commit-slab.h\"\n #include \"shallow.h\"\n-#include \"string-list.h\"\n #include \"json-writer.h\"\n #include \"trace2.h\"\n #include \"tree.h\"\n@@ -201,23 +198,6 @@ static int commit_gen_cmp(const void *va, const void *vb)\n \treturn 0;\n }\n \n-char *get_commit_graph_filename(const char *dir)\n-{\n-\treturn xstrfmt(\"%s/info/commit-graph\", dir);\n-}\n-\n-static char *get_split_graph_filename(const char *dir,\n-\t\t\t\t      const char *oid_hex)\n-{\n-\treturn xstrfmt(\"%s/info/commit-graphs/graph-%s.graph\", dir,\n-\t\t       oid_hex);\n-}\n-\n-char *get_commit_graph_chain_filename(const char *dir)\n-{\n-\treturn xstrfmt(\"%s/info/commit-graphs/commit-graph-chain\", dir);\n-}\n-\n static struct commit_graph *alloc_commit_graph(void)\n {\n \tstruct commit_graph *g = xcalloc(1, sizeof(*g));\n@@ -246,18 +226,6 @@ static int commit_graph_compatible(struct repository *r)\n \treturn 1;\n }\n \n-int open_commit_graph(const char *graph_file, int *fd, struct stat *st)\n-{\n-\t*fd = git_open(graph_file);\n-\tif (*fd < 0)\n-\t\treturn 0;\n-\tif (fstat(*fd, st)) {\n-\t\tclose(*fd);\n-\t\treturn 0;\n-\t}\n-\treturn 1;\n-}\n-\n struct commit_graph *commit_graph_read(struct repository *repo,\n \t\t\t\t       int fd, struct stat *st)\n {\n@@ -494,37 +462,6 @@ struct commit_graph *commit_graph_parse(struct repository *r,\n \treturn NULL;\n }\n \n-static struct commit_graph *load_commit_graph_one(struct repository *repo,\n-\t\t\t\t\t\t  const char *dir,\n-\t\t\t\t\t\t  const char *graph_file)\n-{\n-\tstruct stat st;\n-\tint fd;\n-\tstruct commit_graph *g;\n-\tint open_ok = open_commit_graph(graph_file, &fd, &st);\n-\n-\tif (!open_ok)\n-\t\treturn NULL;\n-\n-\tg = commit_graph_read(repo, fd, &st);\n-\tif (g) {\n-\t\tg->dir = absolute_pathdup(dir);\n-\t\tg->filename = absolute_pathdup(graph_file);\n-\t}\n-\n-\treturn g;\n-}\n-\n-static struct commit_graph *load_commit_graph_v1(struct repository *repo,\n-\t\t\t\t\t\t const char *dir)\n-{\n-\tchar *graph_name = get_commit_graph_filename(dir);\n-\tstruct commit_graph *g = load_commit_graph_one(repo, dir, graph_name);\n-\tfree(graph_name);\n-\n-\treturn g;\n-}\n-\n /*\n  * returns 1 if and only if all graphs in the chain have\n  * corrected commit dates stored in the generation_data chunk.\n@@ -625,120 +562,6 @@ int commit_graph_link_base(struct commit_graph *g,\n \treturn 0;\n }\n \n-int open_commit_graph_chain(const char *chain_file,\n-\t\t\t    int *fd, struct stat *st,\n-\t\t\t    const struct git_hash_algo *hash_algo)\n-{\n-\t*fd = git_open(chain_file);\n-\tif (*fd < 0)\n-\t\treturn 0;\n-\tif (fstat(*fd, st)) {\n-\t\tclose(*fd);\n-\t\treturn 0;\n-\t}\n-\tif (st->st_size < hash_algo->hexsz) {\n-\t\tclose(*fd);\n-\t\tif (!st->st_size) {\n-\t\t\t/* treat empty files the same as missing */\n-\t\t\terrno = ENOENT;\n-\t\t} else {\n-\t\t\twarning(_(\"commit-graph chain file too small\"));\n-\t\t\terrno = EINVAL;\n-\t\t}\n-\t\treturn 0;\n-\t}\n-\treturn 1;\n-}\n-\n-struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n-\t\t\t\t\t\t   int fd, struct stat *st)\n-{\n-\tstruct commit_graph *graph_chain = NULL;\n-\tstruct strbuf line = STRBUF_INIT;\n-\tint i = 0, valid = 1, count;\n-\tFILE *fp = xfdopen(fd, \"r\");\n-\n-\tcount = st->st_size / (odb->repo->hash_algo->hexsz + 1);\n-\n-\tfor (i = 0; i < count; i++) {\n-\t\tstruct odb_source_files *files;\n-\t\tstruct object_id oid;\n-\n-\t\tif (strbuf_getline_lf(&line, fp) == EOF)\n-\t\t\tbreak;\n-\n-\t\tif (get_oid_hex_algop(line.buf, &oid, odb->repo->hash_algo)) {\n-\t\t\twarning(_(\"invalid commit-graph chain: line '%s' not a hash\"),\n-\t\t\t\tline.buf);\n-\t\t\tvalid = 0;\n-\t\t\tbreak;\n-\t\t}\n-\n-\t\tvalid = 0;\n-\n-\t\tfiles = odb_source_files_downcast(odb->source);\n-\t\tfor (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {\n-\t\t\tchar *graph_name = get_split_graph_filename(dir->abspath, line.buf);\n-\t\t\tstruct commit_graph *g = load_commit_graph_one(odb->repo, dir->abspath, graph_name);\n-\n-\t\t\tfree(graph_name);\n-\n-\t\t\tif (g) {\n-\t\t\t\tif (!commit_graph_link_base(g, graph_chain)) {\n-\t\t\t\t\tgraph_chain = g;\n-\t\t\t\t\tvalid = 1;\n-\t\t\t\t} else {\n-\t\t\t\t\tcommit_graph_free(g);\n-\t\t\t\t}\n-\n-\t\t\t\tbreak;\n-\t\t\t}\n-\t\t}\n-\n-\t\tif (!valid) {\n-\t\t\twarning(_(\"unable to find all commit-graph files\"));\n-\t\t\tbreak;\n-\t\t}\n-\t}\n-\n-\tcommit_graph_validate_chain(graph_chain);\n-\tif (graph_chain)\n-\t\tgraph_chain->incomplete = !valid;\n-\n-\tfclose(fp);\n-\tstrbuf_release(&line);\n-\n-\treturn graph_chain;\n-}\n-\n-static struct commit_graph *load_commit_graph_chain(struct repository *repo,\n-\t\t\t\t\t\t    const char *dir)\n-{\n-\tchar *chain_file = get_commit_graph_chain_filename(dir);\n-\tstruct stat st;\n-\tint fd;\n-\tstruct commit_graph *g = NULL;\n-\n-\tif (open_commit_graph_chain(chain_file, &fd, &st, repo->hash_algo)) {\n-\t\t/* ownership of fd is taken over by load function */\n-\t\tg = load_commit_graph_chain_fd_st(repo->objects, fd, &st);\n-\t}\n-\n-\tfree(chain_file);\n-\treturn g;\n-}\n-\n-struct commit_graph *read_commit_graph_one(struct repository *repo,\n-\t\t\t\t\t   const char *dir)\n-{\n-\tstruct commit_graph *g = load_commit_graph_v1(repo, dir);\n-\n-\tif (!g)\n-\t\tg = load_commit_graph_chain(repo, dir);\n-\n-\treturn g;\n-}\n-\n /*\n  * Return 1 if commit_graph is non-NULL, and 0 otherwise.\n  *\n@@ -2078,12 +1901,6 @@ static int write_graph_chunk_base(struct hashfile *f,\n \treturn 0;\n }\n \n-static void mark_commit_graphs(struct commit_graph *tip,\n-\t\t\t       struct commit_graph *base);\n-static void expire_commit_graphs(const char *dirpath,\n-\t\t\t\t struct string_list *kept,\n-\t\t\t\t timestamp_t expire_time);\n-\n static int write_commit_graph_file(struct write_commit_graph_context *ctx,\n \t\t\t\t   int fd, const char *path,\n \t\t\t\t   unsigned char *file_hash)\n@@ -2160,200 +1977,6 @@ static int write_commit_graph_file(struct write_commit_graph_context *ctx,\n \treturn 0;\n }\n \n-int commit_graph_write(struct repository *r,\n-\t\t       const char *dir,\n-\t\t       const struct commit_graph_write_options *opts)\n-{\n-\tstruct string_list chain_hashes = STRING_LIST_INIT_DUP;\n-\tstruct string_list kept = STRING_LIST_INIT_DUP;\n-\tstruct tempfile *graph_layer = NULL;\n-\tstruct lock_file lk = LOCK_INIT;\n-\tstruct commit_graph *g, *chain, *base = NULL;\n-\tchar *base_rename_from = NULL, *base_rename_to = NULL;\n-\tchar *graph_name = NULL, *abs_dir = NULL, *new_path = NULL;\n-\tsize_t mergeable_layers = 0, chainable_layers = 0;\n-\tunsigned char hash[GIT_MAX_RAWSZ];\n-\tbool mergeable = true;\n-\tconst char *out_path;\n-\tint out_fd;\n-\tint ret;\n-\n-\tif (!commit_graph_write_allowed(r))\n-\t\treturn 0;\n-\n-\tabs_dir = absolute_pathdup(dir);\n-\n-\tg = prepare_commit_graph(r);\n-\n-\t/*\n-\t * We only consider merging layers that belong to our own object\n-\t * directory. Furthermore, we can only chain on top of either:\n-\t *\n-\t *   - A split commit graph layer, regardless of whether or not it is\n-\t *     local.\n-\t *\n-\t *   - A local non-split commit graph, in which case we'll convert that\n-\t *     layer into a split commit graph layer.\n-\t */\n-\tfor (struct commit_graph *layer = g; layer; layer = layer->base_graph) {\n-\t\tchar *path = get_commit_graph_filename(layer->dir);\n-\t\tbool is_split = !!strcmp(layer->filename, path);\n-\t\tbool is_local = !strcmp(layer->dir, abs_dir);\n-\n-\t\tfree(path);\n-\n-\t\tif (!is_local && !is_split)\n-\t\t\tbreak;\n-\t\tif (!is_local)\n-\t\t\tmergeable = false;\n-\n-\t\tif (mergeable)\n-\t\t\tmergeable_layers++;\n-\t\tchainable_layers++;\n-\t}\n-\n-\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tgraph_name = xstrfmt(\"%s/info/commit-graphs/tmp_graph_XXXXXX\", abs_dir);\n-\t} else {\n-\t\tgraph_name = get_commit_graph_filename(abs_dir);\n-\t}\n-\n-\tif (safe_create_leading_directories(r, graph_name)) {\n-\t\tret = error(_(\"unable to create leading directories of %s\"),\n-\t\t\t    graph_name);\n-\t\tgoto out;\n-\t}\n-\n-\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tchar *chain_name = get_commit_graph_chain_filename(abs_dir);\n-\n-\t\trepo_hold_lock_file_for_update_mode(r, &lk, chain_name,\n-\t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n-\t\tfree(chain_name);\n-\n-\t\tgraph_layer = mks_tempfile_m(graph_name, 0444);\n-\t\tif (!graph_layer) {\n-\t\t\tret = error(_(\"unable to create temporary graph layer\"));\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\tif (adjust_shared_perm(r, get_tempfile_path(graph_layer))) {\n-\t\t\tret = error(_(\"unable to adjust shared permissions for '%s'\"),\n-\t\t\t\t    get_tempfile_path(graph_layer));\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\tout_fd = get_tempfile_fd(graph_layer);\n-\t\tout_path = get_tempfile_path(graph_layer);\n-\t} else {\n-\t\trepo_hold_lock_file_for_update_mode(r, &lk, graph_name,\n-\t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n-\t\tout_fd = get_lock_file_fd(&lk);\n-\t\tout_path = get_lock_file_path(&lk);\n-\t}\n-\n-\tret = commit_graph_write_layer(r, opts, mergeable_layers, chainable_layers,\n-\t\t\t\t       out_fd, out_path, &base, hash);\n-\tif (ret <= 0)\n-\t\tgoto out;\n-\n-\t/*\n-\t * Snapshot everything we need from the current chain before closing\n-\t * it, as we must not have any layers mapped when replacing them.\n-\t */\n-\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tconst char *new_hash = hash_to_hex_algop(hash, r->hash_algo);\n-\n-\t\tnew_path = get_split_graph_filename(abs_dir, new_hash);\n-\n-\t\tstring_list_append(&chain_hashes, new_hash);\n-\t\tfor (chain = base; chain; chain = chain->base_graph)\n-\t\t\tstring_list_append(&chain_hashes, oid_to_hex(&chain->oid));\n-\n-\t\t/*\n-\t\t * In case we have a non-split commit graph we need to convert\n-\t\t * it into the base layer of the new split commit graph by\n-\t\t * renaming it.\n-\t\t */\n-\t\tif (base) {\n-\t\t\tbase_rename_to = get_split_graph_filename(base->dir, oid_to_hex(&base->oid));\n-\t\t\tif (strcmp(base->filename, base_rename_to))\n-\t\t\t\tbase_rename_from = xstrdup(base->filename);\n-\t\t\telse\n-\t\t\t\tFREE_AND_NULL(base_rename_to);\n-\t\t}\n-\n-\t\tfor (chain = base; chain; chain = chain->base_graph) {\n-\t\t\tif (chain == base && base_rename_to)\n-\t\t\t\tstring_list_append(&kept, base_rename_to);\n-\t\t\telse\n-\t\t\t\tstring_list_append(&kept, chain->filename);\n-\t\t}\n-\t\tstring_list_append(&kept, new_path);\n-\n-\t\tmark_commit_graphs(g, base);\n-\t}\n-\n-\tcommit_graph_close(r->objects);\n-\n-\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tFILE *chainf = fdopen_lock_file(&lk, \"w\");\n-\n-\t\tif (!chainf) {\n-\t\t\tret = error(_(\"unable to open commit-graph chain file\"));\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\tif (base_rename_from) {\n-\t\t\tif (rename(base_rename_from, base_rename_to)) {\n-\t\t\t\tret = error(_(\"failed to rename base commit-graph file\"));\n-\t\t\t\tgoto out;\n-\t\t\t}\n-\t\t} else if (!base) {\n-\t\t\tchar *graph_name = get_commit_graph_filename(abs_dir);\n-\t\t\tunlink(graph_name);\n-\t\t\tfree(graph_name);\n-\t\t}\n-\n-\t\tif (rename_tempfile(&graph_layer, new_path)) {\n-\t\t\tret = error(_(\"failed to rename temporary commit-graph file\"));\n-\t\t\tgoto out;\n-\t\t}\n-\n-\t\t/*\n-\t\t * Chain hashes are ordered newest-first, but the chain file is\n-\t\t * oldest-first so we iterate in reverse.\n-\t\t */\n-\t\tfor (size_t i = chain_hashes.nr; i; i--)\n-\t\t\tfprintf(chainf, \"%s\\n\", chain_hashes.items[i - 1].string);\n-\t}\n-\n-\tcommit_lock_file(&lk);\n-\n-\tif (!(opts->flags & COMMIT_GRAPH_WRITE_SPLIT)) {\n-\t\tchar *chain_name = get_commit_graph_chain_filename(abs_dir);\n-\t\tunlink(chain_name);\n-\t\tfree(chain_name);\n-\t}\n-\n-\texpire_commit_graphs(abs_dir, &kept,\n-\t\t\t     opts->expire_time ? opts->expire_time : time(NULL));\n-\n-\tret = 0;\n-\n-out:\n-\trollback_lock_file(&lk);\n-\tdelete_tempfile(&graph_layer);\n-\tstring_list_clear(&chain_hashes, 0);\n-\tstring_list_clear(&kept, 0);\n-\tfree(base_rename_from);\n-\tfree(base_rename_to);\n-\tfree(graph_name);\n-\tfree(new_path);\n-\tfree(abs_dir);\n-\treturn ret;\n-}\n-\n static void split_graph_merge_strategy(struct write_commit_graph_context *ctx,\n \t\t\t\t       struct commit_graph *graph_to_merge)\n {\n@@ -2521,66 +2144,6 @@ static void merge_commit_graphs(struct write_commit_graph_context *ctx,\n \tsort_and_scan_merged_commits(ctx);\n }\n \n-static void mark_commit_graphs(struct commit_graph *tip,\n-\t\t\t       struct commit_graph *base)\n-{\n-\ttime_t now = time(NULL);\n-\n-\tfor (struct commit_graph *g = tip; g && g != base; g = g->base_graph) {\n-\t\tstruct stat st;\n-\t\tstruct utimbuf updated_time;\n-\n-\t\tif (stat(g->filename, &st) < 0)\n-\t\t\tcontinue;\n-\n-\t\tupdated_time.actime = st.st_atime;\n-\t\tupdated_time.modtime = now;\n-\t\tutime(g->filename, &updated_time);\n-\t}\n-}\n-\n-static void expire_commit_graphs(const char *dirpath,\n-\t\t\t\t struct string_list *kept,\n-\t\t\t\t timestamp_t expire_time)\n-{\n-\tstruct strbuf path = STRBUF_INIT;\n-\tDIR *dir;\n-\tstruct dirent *de;\n-\tsize_t dirnamelen;\n-\n-\tstrbuf_addstr(&path, dirpath);\n-\tstrbuf_addstr(&path, \"/info/commit-graphs\");\n-\tdir = opendir(path.buf);\n-\n-\tif (!dir)\n-\t\tgoto out;\n-\n-\tstrbuf_addch(&path, '/');\n-\tdirnamelen = path.len;\n-\twhile ((de = readdir(dir)) != NULL) {\n-\t\tstruct stat st;\n-\n-\t\tstrbuf_setlen(&path, dirnamelen);\n-\t\tstrbuf_addstr(&path, de->d_name);\n-\n-\t\tif (stat(path.buf, &st) < 0)\n-\t\t\tcontinue;\n-\n-\t\tif (st.st_mtime > expire_time)\n-\t\t\tcontinue;\n-\t\tif (path.len < 6 || strcmp(path.buf + path.len - 6, \".graph\"))\n-\t\t\tcontinue;\n-\n-\t\tif (!unsorted_string_list_has_string(kept, path.buf))\n-\t\t\tunlink(path.buf);\n-\t}\n-\n-out:\n-\tif (dir)\n-\t\tclosedir(dir);\n-\tstrbuf_release(&path);\n-}\n-\n bool commit_graph_write_allowed(struct repository *r)\n {\n \tprepare_repo_settings(r);\ndiff --git a/commit-graph.h b/commit-graph.h\nindex 3b5e0eb2d73..3295498fd34 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -141,8 +141,6 @@ int commit_graph_link_base(struct commit_graph *g,\n  * filters in case they aren't.\n  */\n void commit_graph_validate_chain(struct commit_graph *g);\n-struct commit_graph *read_commit_graph_one(struct repository *repo,\n-\t\t\t\t\t   const char *dir);\n \n struct repo_settings;\n \ndiff --git a/odb/source-files.c b/odb/source-files.c\nindex 5ad27fd4bc7..90cda52b48b 100644\n--- a/odb/source-files.c\n+++ b/odb/source-files.c\n@@ -1231,6 +1231,180 @@ static int odb_source_files_generate_pack(struct odb_source *source UNUSED,\n \treturn 0;\n }\n \n+char *get_commit_graph_filename(const char *dir)\n+{\n+\treturn xstrfmt(\"%s/info/commit-graph\", dir);\n+}\n+\n+static char *get_split_graph_filename(const char *dir,\n+\t\t\t\t      const char *oid_hex)\n+{\n+\treturn xstrfmt(\"%s/info/commit-graphs/graph-%s.graph\", dir,\n+\t\t       oid_hex);\n+}\n+\n+char *get_commit_graph_chain_filename(const char *dir)\n+{\n+\treturn xstrfmt(\"%s/info/commit-graphs/commit-graph-chain\", dir);\n+}\n+\n+int open_commit_graph(const char *graph_file, int *fd, struct stat *st)\n+{\n+\t*fd = git_open(graph_file);\n+\tif (*fd < 0)\n+\t\treturn 0;\n+\tif (fstat(*fd, st)) {\n+\t\tclose(*fd);\n+\t\treturn 0;\n+\t}\n+\treturn 1;\n+}\n+\n+static struct commit_graph *load_commit_graph_one(struct repository *repo,\n+\t\t\t\t\t\t  const char *dir,\n+\t\t\t\t\t\t  const char *graph_file)\n+{\n+\tstruct stat st;\n+\tint fd;\n+\tstruct commit_graph *g;\n+\tint open_ok = open_commit_graph(graph_file, &fd, &st);\n+\n+\tif (!open_ok)\n+\t\treturn NULL;\n+\n+\tg = commit_graph_read(repo, fd, &st);\n+\tif (g) {\n+\t\tg->dir = absolute_pathdup(dir);\n+\t\tg->filename = absolute_pathdup(graph_file);\n+\t}\n+\n+\treturn g;\n+}\n+\n+static struct commit_graph *load_commit_graph_v1(struct repository *repo,\n+\t\t\t\t\t\t const char *dir)\n+{\n+\tchar *graph_name = get_commit_graph_filename(dir);\n+\tstruct commit_graph *g = load_commit_graph_one(repo, dir, graph_name);\n+\tfree(graph_name);\n+\n+\treturn g;\n+}\n+\n+int open_commit_graph_chain(const char *chain_file,\n+\t\t\t    int *fd, struct stat *st,\n+\t\t\t    const struct git_hash_algo *hash_algo)\n+{\n+\t*fd = git_open(chain_file);\n+\tif (*fd < 0)\n+\t\treturn 0;\n+\tif (fstat(*fd, st)) {\n+\t\tclose(*fd);\n+\t\treturn 0;\n+\t}\n+\tif (st->st_size < (ssize_t) hash_algo->hexsz) {\n+\t\tclose(*fd);\n+\t\tif (!st->st_size) {\n+\t\t\t/* treat empty files the same as missing */\n+\t\t\terrno = ENOENT;\n+\t\t} else {\n+\t\t\twarning(_(\"commit-graph chain file too small\"));\n+\t\t\terrno = EINVAL;\n+\t\t}\n+\t\treturn 0;\n+\t}\n+\treturn 1;\n+}\n+\n+struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n+\t\t\t\t\t\t   int fd, struct stat *st)\n+{\n+\tstruct commit_graph *graph_chain = NULL;\n+\tstruct strbuf line = STRBUF_INIT;\n+\tint i = 0, valid = 1, count;\n+\tFILE *fp = xfdopen(fd, \"r\");\n+\n+\tcount = st->st_size / (odb->repo->hash_algo->hexsz + 1);\n+\n+\tfor (i = 0; i < count; i++) {\n+\t\tstruct odb_source_files *files;\n+\t\tstruct object_id oid;\n+\n+\t\tif (strbuf_getline_lf(&line, fp) == EOF)\n+\t\t\tbreak;\n+\n+\t\tif (get_oid_hex_algop(line.buf, &oid, odb->repo->hash_algo)) {\n+\t\t\twarning(_(\"invalid commit-graph chain: line '%s' not a hash\"),\n+\t\t\t\tline.buf);\n+\t\t\tvalid = 0;\n+\t\t\tbreak;\n+\t\t}\n+\n+\t\tvalid = 0;\n+\n+\t\tfiles = odb_source_files_downcast(odb->source);\n+\t\tfor (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {\n+\t\t\tchar *graph_name = get_split_graph_filename(dir->abspath, line.buf);\n+\t\t\tstruct commit_graph *g = load_commit_graph_one(odb->repo, dir->abspath, graph_name);\n+\n+\t\t\tfree(graph_name);\n+\n+\t\t\tif (g) {\n+\t\t\t\tif (!commit_graph_link_base(g, graph_chain)) {\n+\t\t\t\t\tgraph_chain = g;\n+\t\t\t\t\tvalid = 1;\n+\t\t\t\t} else {\n+\t\t\t\t\tcommit_graph_free(g);\n+\t\t\t\t}\n+\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t}\n+\n+\t\tif (!valid) {\n+\t\t\twarning(_(\"unable to find all commit-graph files\"));\n+\t\t\tbreak;\n+\t\t}\n+\t}\n+\n+\tcommit_graph_validate_chain(graph_chain);\n+\tif (graph_chain)\n+\t\tgraph_chain->incomplete = !valid;\n+\n+\tfclose(fp);\n+\tstrbuf_release(&line);\n+\n+\treturn graph_chain;\n+}\n+\n+static struct commit_graph *load_commit_graph_chain(struct repository *repo,\n+\t\t\t\t\t\t    const char *dir)\n+{\n+\tchar *chain_file = get_commit_graph_chain_filename(dir);\n+\tstruct stat st;\n+\tint fd;\n+\tstruct commit_graph *g = NULL;\n+\n+\tif (open_commit_graph_chain(chain_file, &fd, &st, repo->hash_algo)) {\n+\t\t/* ownership of fd is taken over by load function */\n+\t\tg = load_commit_graph_chain_fd_st(repo->objects, fd, &st);\n+\t}\n+\n+\tfree(chain_file);\n+\treturn g;\n+}\n+\n+static struct commit_graph *read_commit_graph_one(struct repository *repo,\n+\t\t\t\t\t\t  const char *dir)\n+{\n+\tstruct commit_graph *g = load_commit_graph_v1(repo, dir);\n+\n+\tif (!g)\n+\t\tg = load_commit_graph_chain(repo, dir);\n+\n+\treturn g;\n+}\n+\n static int odb_source_files_read_commit_graph(struct odb_source *source,\n \t\t\t\t\t      struct commit_graph **out)\n {\n@@ -1248,6 +1422,260 @@ static int odb_source_files_read_commit_graph(struct odb_source *source,\n \treturn 1;\n }\n \n+static void mark_commit_graphs(struct commit_graph *tip,\n+\t\t\t       struct commit_graph *base)\n+{\n+\ttime_t now = time(NULL);\n+\n+\tfor (struct commit_graph *g = tip; g && g != base; g = g->base_graph) {\n+\t\tstruct stat st;\n+\t\tstruct utimbuf updated_time;\n+\n+\t\tif (stat(g->filename, &st) < 0)\n+\t\t\tcontinue;\n+\n+\t\tupdated_time.actime = st.st_atime;\n+\t\tupdated_time.modtime = now;\n+\t\tutime(g->filename, &updated_time);\n+\t}\n+}\n+\n+static void expire_commit_graphs(const char *dirpath,\n+\t\t\t\t struct string_list *kept,\n+\t\t\t\t timestamp_t expire_time)\n+{\n+\tstruct strbuf path = STRBUF_INIT;\n+\tDIR *dir;\n+\tstruct dirent *de;\n+\tsize_t dirnamelen;\n+\n+\tstrbuf_addstr(&path, dirpath);\n+\tstrbuf_addstr(&path, \"/info/commit-graphs\");\n+\tdir = opendir(path.buf);\n+\n+\tif (!dir)\n+\t\tgoto out;\n+\n+\tstrbuf_addch(&path, '/');\n+\tdirnamelen = path.len;\n+\twhile ((de = readdir(dir)) != NULL) {\n+\t\tstruct stat st;\n+\n+\t\tstrbuf_setlen(&path, dirnamelen);\n+\t\tstrbuf_addstr(&path, de->d_name);\n+\n+\t\tif (stat(path.buf, &st) < 0)\n+\t\t\tcontinue;\n+\n+\t\tif ((timestamp_t) st.st_mtime > expire_time)\n+\t\t\tcontinue;\n+\t\tif (path.len < 6 || strcmp(path.buf + path.len - 6, \".graph\"))\n+\t\t\tcontinue;\n+\n+\t\tif (!unsorted_string_list_has_string(kept, path.buf))\n+\t\t\tunlink(path.buf);\n+\t}\n+\n+out:\n+\tif (dir)\n+\t\tclosedir(dir);\n+\tstrbuf_release(&path);\n+}\n+\n+int commit_graph_write(struct repository *r,\n+\t\t       const char *dir,\n+\t\t       const struct commit_graph_write_options *opts)\n+{\n+\tstruct string_list chain_hashes = STRING_LIST_INIT_DUP;\n+\tstruct string_list kept = STRING_LIST_INIT_DUP;\n+\tstruct tempfile *graph_layer = NULL;\n+\tstruct lock_file lk = LOCK_INIT;\n+\tstruct commit_graph *g, *chain, *base = NULL;\n+\tchar *base_rename_from = NULL, *base_rename_to = NULL;\n+\tchar *graph_name = NULL, *abs_dir = NULL, *new_path = NULL;\n+\tsize_t mergeable_layers = 0, chainable_layers = 0;\n+\tunsigned char hash[GIT_MAX_RAWSZ];\n+\tbool mergeable = true;\n+\tconst char *out_path;\n+\tint out_fd;\n+\tint ret;\n+\n+\tif (!commit_graph_write_allowed(r))\n+\t\treturn 0;\n+\n+\tabs_dir = absolute_pathdup(dir);\n+\n+\tg = prepare_commit_graph(r);\n+\n+\t/*\n+\t * We only consider merging layers that belong to our own object\n+\t * directory. Furthermore, we can only chain on top of either:\n+\t *\n+\t *   - A split commit graph layer, regardless of whether or not it is\n+\t *     local.\n+\t *\n+\t *   - A local non-split commit graph, in which case we'll convert that\n+\t *     layer into a split commit graph layer.\n+\t */\n+\tfor (struct commit_graph *layer = g; layer; layer = layer->base_graph) {\n+\t\tchar *path = get_commit_graph_filename(layer->dir);\n+\t\tbool is_split = !!strcmp(layer->filename, path);\n+\t\tbool is_local = !strcmp(layer->dir, abs_dir);\n+\n+\t\tfree(path);\n+\n+\t\tif (!is_local && !is_split)\n+\t\t\tbreak;\n+\t\tif (!is_local)\n+\t\t\tmergeable = false;\n+\n+\t\tif (mergeable)\n+\t\t\tmergeable_layers++;\n+\t\tchainable_layers++;\n+\t}\n+\n+\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\t\tgraph_name = xstrfmt(\"%s/info/commit-graphs/tmp_graph_XXXXXX\", abs_dir);\n+\t} else {\n+\t\tgraph_name = get_commit_graph_filename(abs_dir);\n+\t}\n+\n+\tif (safe_create_leading_directories(r, graph_name)) {\n+\t\tret = error(_(\"unable to create leading directories of %s\"),\n+\t\t\t    graph_name);\n+\t\tgoto out;\n+\t}\n+\n+\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\t\tchar *chain_name = get_commit_graph_chain_filename(abs_dir);\n+\n+\t\trepo_hold_lock_file_for_update_mode(r, &lk, chain_name,\n+\t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n+\t\tfree(chain_name);\n+\n+\t\tgraph_layer = mks_tempfile_m(graph_name, 0444);\n+\t\tif (!graph_layer) {\n+\t\t\tret = error(_(\"unable to create temporary graph layer\"));\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (adjust_shared_perm(r, get_tempfile_path(graph_layer))) {\n+\t\t\tret = error(_(\"unable to adjust shared permissions for '%s'\"),\n+\t\t\t\t    get_tempfile_path(graph_layer));\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tout_fd = get_tempfile_fd(graph_layer);\n+\t\tout_path = get_tempfile_path(graph_layer);\n+\t} else {\n+\t\trepo_hold_lock_file_for_update_mode(r, &lk, graph_name,\n+\t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n+\t\tout_fd = get_lock_file_fd(&lk);\n+\t\tout_path = get_lock_file_path(&lk);\n+\t}\n+\n+\tret = commit_graph_write_layer(r, opts, mergeable_layers, chainable_layers,\n+\t\t\t\t       out_fd, out_path, &base, hash);\n+\tif (ret <= 0)\n+\t\tgoto out;\n+\n+\t/*\n+\t * Snapshot everything we need from the current chain before closing\n+\t * it, as we must not have any layers mapped when replacing them.\n+\t */\n+\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\t\tconst char *new_hash = hash_to_hex_algop(hash, r->hash_algo);\n+\n+\t\tnew_path = get_split_graph_filename(abs_dir, new_hash);\n+\n+\t\tstring_list_append(&chain_hashes, new_hash);\n+\t\tfor (chain = base; chain; chain = chain->base_graph)\n+\t\t\tstring_list_append(&chain_hashes, oid_to_hex(&chain->oid));\n+\n+\t\t/*\n+\t\t * In case we have a non-split commit graph we need to convert\n+\t\t * it into the base layer of the new split commit graph by\n+\t\t * renaming it.\n+\t\t */\n+\t\tif (base) {\n+\t\t\tbase_rename_to = get_split_graph_filename(base->dir, oid_to_hex(&base->oid));\n+\t\t\tif (strcmp(base->filename, base_rename_to))\n+\t\t\t\tbase_rename_from = xstrdup(base->filename);\n+\t\t\telse\n+\t\t\t\tFREE_AND_NULL(base_rename_to);\n+\t\t}\n+\n+\t\tfor (chain = base; chain; chain = chain->base_graph) {\n+\t\t\tif (chain == base && base_rename_to)\n+\t\t\t\tstring_list_append(&kept, base_rename_to);\n+\t\t\telse\n+\t\t\t\tstring_list_append(&kept, chain->filename);\n+\t\t}\n+\t\tstring_list_append(&kept, new_path);\n+\n+\t\tmark_commit_graphs(g, base);\n+\t}\n+\n+\tcommit_graph_close(r->objects);\n+\n+\tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n+\t\tFILE *chainf = fdopen_lock_file(&lk, \"w\");\n+\n+\t\tif (!chainf) {\n+\t\t\tret = error(_(\"unable to open commit-graph chain file\"));\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\tif (base_rename_from) {\n+\t\t\tif (rename(base_rename_from, base_rename_to)) {\n+\t\t\t\tret = error(_(\"failed to rename base commit-graph file\"));\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t} else if (!base) {\n+\t\t\tchar *graph_name = get_commit_graph_filename(abs_dir);\n+\t\t\tunlink(graph_name);\n+\t\t\tfree(graph_name);\n+\t\t}\n+\n+\t\tif (rename_tempfile(&graph_layer, new_path)) {\n+\t\t\tret = error(_(\"failed to rename temporary commit-graph file\"));\n+\t\t\tgoto out;\n+\t\t}\n+\n+\t\t/*\n+\t\t * Chain hashes are ordered newest-first, but the chain file is\n+\t\t * oldest-first so we iterate in reverse.\n+\t\t */\n+\t\tfor (size_t i = chain_hashes.nr; i; i--)\n+\t\t\tfprintf(chainf, \"%s\\n\", chain_hashes.items[i - 1].string);\n+\t}\n+\n+\tcommit_lock_file(&lk);\n+\n+\tif (!(opts->flags & COMMIT_GRAPH_WRITE_SPLIT)) {\n+\t\tchar *chain_name = get_commit_graph_chain_filename(abs_dir);\n+\t\tunlink(chain_name);\n+\t\tfree(chain_name);\n+\t}\n+\n+\texpire_commit_graphs(abs_dir, &kept,\n+\t\t\t     opts->expire_time ? opts->expire_time : (timestamp_t) time(NULL));\n+\n+\tret = 0;\n+\n+out:\n+\trollback_lock_file(&lk);\n+\tdelete_tempfile(&graph_layer);\n+\tstring_list_clear(&chain_hashes, 0);\n+\tstring_list_clear(&kept, 0);\n+\tfree(base_rename_from);\n+\tfree(base_rename_to);\n+\tfree(graph_name);\n+\tfree(new_path);\n+\tfree(abs_dir);\n+\treturn ret;\n+}\n+\n static int odb_source_files_write_commit_graph(struct odb_source *source,\n \t\t\t\t\t       const struct commit_graph_write_options *opts)\n {\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558547","messageId":"20261116-pks-odb-pluggable-commit-graphs-v1-12-a83fc1ff0808@pks.im","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-0-a83fc1ff0808@pks.im","subject":"[PATCH 12/12] odb/source-files: internalize logic to derive commit graph paths","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-11-16T13:07:48Z","receivedAt":"2026-11-16T13:07:48Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"In the preceding commit we have moved the logic to derive commit graph\npaths into the \"files\" backend. We couldn't yet make the logic private\nto the backend though because we still use it when we verify commit\ngraphs.\n\nRefactor the code and expose two new functions that are specific to the\n\"files\" backend to read and write a commit graph into a specific object\ndirectory. These functions are only for use for existing callers in\ngit-commit-graph(1), so adapt that one caller to use those functions.\n\nThis converts the last caller outside of the \"files\" backend to stop\ncaring about exact commit graph paths anymore. Drop the declarations for\nfunctions that derive such paths and make them private.\n\nSigned-off-by: Patrick Steinhardt <ps@pks.im>\n---\n builtin/commit-graph.c |  57 +++++++-----------\n commit-graph.h         |  21 -------\n odb/source-files.c     | 157 ++++++++++++++++++++++++++++---------------------\n odb/source-files.h     |  12 ++++\n 4 files changed, 123 insertions(+), 124 deletions(-)\n\ndiff --git a/builtin/commit-graph.c b/builtin/commit-graph.c\nindex 6ee644d8487..d9d3cdd335a 100644\n--- a/builtin/commit-graph.c\n+++ b/builtin/commit-graph.c\n@@ -59,13 +59,7 @@ static int cmd_commit_graph_verify(int argc,\n {\n \tstruct opts_commit_graph opts = { 0 };\n \tstruct commit_graph *graph = NULL;\n-\tstruct odb_files_dir *dir = NULL;\n-\tchar *graph_name;\n-\tchar *chain_name;\n-\tenum { OPENED_NONE, OPENED_GRAPH, OPENED_CHAIN } opened = OPENED_NONE;\n \tenum commit_graph_verify_flags flags = isatty(2) ? COMMIT_GRAPH_VERIFY_PROGRESS : 0;\n-\tint fd;\n-\tstruct stat st;\n \tint ret;\n \tstruct option options[] = {\n \t\tOPT_BIT(0, \"shallow\", &flags,\n@@ -85,37 +79,27 @@ static int cmd_commit_graph_verify(int argc,\n \tif (argc)\n \t\tusage_with_options(builtin_commit_graph_verify_usage, options);\n \n-\tif (!opts.obj_dir)\n-\t\topts.obj_dir = repo_get_object_directory(repo);\n-\n-\tdir = odb_source_files_find_dir(repo->objects, opts.obj_dir);\n-\tif (!dir)\n-\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n-\n-\tgraph_name = get_commit_graph_filename(dir->abspath);\n-\tchain_name = get_commit_graph_chain_filename(dir->abspath);\n-\tif (open_commit_graph(graph_name, &fd, &st))\n-\t\topened = OPENED_GRAPH;\n-\telse if (errno != ENOENT)\n-\t\tdie_errno(_(\"Could not open commit-graph '%s'\"), graph_name);\n-\telse if (open_commit_graph_chain(chain_name, &fd, &st,\n-\t\t\t\t\t repo->hash_algo))\n-\t\topened = OPENED_CHAIN;\n-\telse if (errno != ENOENT)\n-\t\tdie_errno(_(\"could not open commit-graph chain '%s'\"), chain_name);\n-\n-\tFREE_AND_NULL(graph_name);\n-\tFREE_AND_NULL(chain_name);\n-\n-\tif (opened == OPENED_NONE)\n-\t\treturn 0;\n-\telse if (opened == OPENED_GRAPH)\n-\t\tgraph = commit_graph_read(repo, fd, &st);\n-\telse\n-\t\tgraph = load_commit_graph_chain_fd_st(repo->objects, fd, &st);\n+\tif (repo->objects->source->type == ODB_SOURCE_FILES) {\n+\t\tstruct odb_source_files *files = odb_source_files_downcast(repo->objects->source);\n+\t\tstruct odb_files_dir *dir = files->dirs;\n+\n+\t\tif (opts.obj_dir) {\n+\t\t\tdir = odb_source_files_find_dir(repo->objects, opts.obj_dir);\n+\t\t\tif (!dir)\n+\t\t\t\tdie(_(\"could not find object directory matching %s\"), opts.obj_dir);\n+\t\t}\n \n-\tif (!graph)\n+\t\tret = odb_source_files_read_commit_graph_dir(files, dir, &graph);\n+\t} else {\n+\t\tif (opts.obj_dir)\n+\t\t\tdie(_(\"--object-dir= is not supported with %s backend\"),\n+\t\t\t    odb_source_type_to_name(repo->objects->source->type));\n+\t\tret = odb_source_read_commit_graph(repo->objects->source, &graph);\n+\t}\n+\tif (ret < 0)\n \t\treturn 1;\n+\tif (ret > 0)\n+\t\treturn 0;\n \n \tret = commit_graph_verify(graph, flags);\n \n@@ -328,7 +312,8 @@ static int cmd_commit_graph_write(int argc,\n \t}\n \n \tif (dir) {\n-\t\tif (commit_graph_write(repo, dir->abspath, &write_opts))\n+\t\tstruct odb_source_files *files = odb_source_files_downcast(repo->objects->source);\n+\t\tif (odb_source_files_write_commit_graph_dir(files, dir, &write_opts))\n \t\t\tresult = 1;\n \t} else {\n \t\tif (odb_write_commit_graph(repo->objects, &write_opts))\ndiff --git a/commit-graph.h b/commit-graph.h\nindex 3295498fd34..a84227c7a6d 100644\n--- a/commit-graph.h\n+++ b/commit-graph.h\n@@ -29,12 +29,7 @@ struct repository;\n struct object_database;\n struct string_list;\n \n-char *get_commit_graph_filename(const char *dir);\n-char *get_commit_graph_chain_filename(const char *dir);\n struct commit_graph *prepare_commit_graph(struct repository *r);\n-int open_commit_graph(const char *graph_file, int *fd, struct stat *st);\n-int open_commit_graph_chain(const char *chain_file, int *fd, struct stat *st,\n-\t\t\t    const struct git_hash_algo *hash_algo);\n \n /*\n  * Given a commit struct, try to fill the commit struct info, including:\n@@ -124,8 +119,6 @@ struct commit_graph {\n  */\n struct commit_graph *commit_graph_read(struct repository *repo,\n \t\t\t\t       int fd, struct stat *st);\n-struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n-\t\t\t\t\t\t   int fd, struct stat *st);\n \n /*\n  * Link the commit graph layer `g` on top of the chain `base`. `base` may be\n@@ -226,20 +219,6 @@ int commit_graph_write_layer(struct repository *r,\n \t\t\t     struct commit_graph **base,\n \t\t\t     unsigned char *hash);\n \n-/*\n- * The commit_graph_write* methods return zero on success\n- * and a negative value on failure. Note that if the repository\n- * is not compatible with the commit-graph feature, then the\n- * methods will return 0 without writing a commit-graph.\n- *\n- * The object directory `dir` may be given as a relative path; it is\n- * canonicalized internally so that it compares equal to the directory of\n- * graphs that have already been loaded.\n- */\n-int commit_graph_write(struct repository *r,\n-\t\t       const char *dir,\n-\t\t       const struct commit_graph_write_options *opts);\n-\n enum commit_graph_verify_flags {\n \tCOMMIT_GRAPH_VERIFY_SHALLOW = (1 << 0),\n \tCOMMIT_GRAPH_VERIFY_PROGRESS = (1 << 1),\ndiff --git a/odb/source-files.c b/odb/source-files.c\nindex 90cda52b48b..16d331e4f28 100644\n--- a/odb/source-files.c\n+++ b/odb/source-files.c\n@@ -1231,7 +1231,7 @@ static int odb_source_files_generate_pack(struct odb_source *source UNUSED,\n \treturn 0;\n }\n \n-char *get_commit_graph_filename(const char *dir)\n+static char *get_commit_graph_filename(const char *dir)\n {\n \treturn xstrfmt(\"%s/info/commit-graph\", dir);\n }\n@@ -1243,12 +1243,12 @@ static char *get_split_graph_filename(const char *dir,\n \t\t       oid_hex);\n }\n \n-char *get_commit_graph_chain_filename(const char *dir)\n+static char *get_commit_graph_chain_filename(const char *dir)\n {\n \treturn xstrfmt(\"%s/info/commit-graphs/commit-graph-chain\", dir);\n }\n \n-int open_commit_graph(const char *graph_file, int *fd, struct stat *st)\n+static int open_commit_graph(const char *graph_file, int *fd, struct stat *st)\n {\n \t*fd = git_open(graph_file);\n \tif (*fd < 0)\n@@ -1260,40 +1260,45 @@ int open_commit_graph(const char *graph_file, int *fd, struct stat *st)\n \treturn 1;\n }\n \n-static struct commit_graph *load_commit_graph_one(struct repository *repo,\n-\t\t\t\t\t\t  const char *dir,\n-\t\t\t\t\t\t  const char *graph_file)\n+static int load_commit_graph_one(struct repository *repo,\n+\t\t\t\t const char *dir,\n+\t\t\t\t const char *graph_file,\n+\t\t\t\t struct commit_graph **out)\n {\n \tstruct stat st;\n \tint fd;\n \tstruct commit_graph *g;\n-\tint open_ok = open_commit_graph(graph_file, &fd, &st);\n \n-\tif (!open_ok)\n-\t\treturn NULL;\n+\tif (!open_commit_graph(graph_file, &fd, &st)) {\n+\t\tif (errno == ENOENT)\n+\t\t\treturn 1;\n+\t\treturn error_errno(_(\"Could not open commit-graph '%s'\"),\n+\t\t\t\t   graph_file);\n+\t}\n \n \tg = commit_graph_read(repo, fd, &st);\n-\tif (g) {\n-\t\tg->dir = absolute_pathdup(dir);\n-\t\tg->filename = absolute_pathdup(graph_file);\n-\t}\n+\tif (!g)\n+\t\treturn -1;\n \n-\treturn g;\n+\tg->dir = absolute_pathdup(dir);\n+\tg->filename = absolute_pathdup(graph_file);\n+\t*out = g;\n+\treturn 0;\n }\n \n-static struct commit_graph *load_commit_graph_v1(struct repository *repo,\n-\t\t\t\t\t\t const char *dir)\n+static int load_commit_graph_v1(struct repository *repo,\n+\t\t\t\tconst char *dir,\n+\t\t\t\tstruct commit_graph **out)\n {\n \tchar *graph_name = get_commit_graph_filename(dir);\n-\tstruct commit_graph *g = load_commit_graph_one(repo, dir, graph_name);\n+\tint ret = load_commit_graph_one(repo, dir, graph_name, out);\n \tfree(graph_name);\n-\n-\treturn g;\n+\treturn ret;\n }\n \n-int open_commit_graph_chain(const char *chain_file,\n-\t\t\t    int *fd, struct stat *st,\n-\t\t\t    const struct git_hash_algo *hash_algo)\n+static int open_commit_graph_chain(const char *chain_file,\n+\t\t\t\t   int *fd, struct stat *st,\n+\t\t\t\t   const struct git_hash_algo *hash_algo)\n {\n \t*fd = git_open(chain_file);\n \tif (*fd < 0)\n@@ -1316,8 +1321,8 @@ int open_commit_graph_chain(const char *chain_file,\n \treturn 1;\n }\n \n-struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n-\t\t\t\t\t\t   int fd, struct stat *st)\n+static struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n+\t\t\t\t\t\t\t  int fd, struct stat *st)\n {\n \tstruct commit_graph *graph_chain = NULL;\n \tstruct strbuf line = STRBUF_INIT;\n@@ -1345,20 +1350,25 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n \t\tfiles = odb_source_files_downcast(odb->source);\n \t\tfor (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {\n \t\t\tchar *graph_name = get_split_graph_filename(dir->abspath, line.buf);\n-\t\t\tstruct commit_graph *g = load_commit_graph_one(odb->repo, dir->abspath, graph_name);\n+\t\t\tstruct commit_graph *g = NULL;\n+\t\t\tint ret;\n \n+\t\t\tret = load_commit_graph_one(odb->repo, dir->abspath, graph_name, &g);\n \t\t\tfree(graph_name);\n \n-\t\t\tif (g) {\n-\t\t\t\tif (!commit_graph_link_base(g, graph_chain)) {\n-\t\t\t\t\tgraph_chain = g;\n-\t\t\t\t\tvalid = 1;\n-\t\t\t\t} else {\n-\t\t\t\t\tcommit_graph_free(g);\n-\t\t\t\t}\n-\n+\t\t\tif (ret > 0)\n+\t\t\t\tcontinue;\n+\t\t\tif (ret < 0)\n \t\t\t\tbreak;\n+\n+\t\t\tif (!commit_graph_link_base(g, graph_chain)) {\n+\t\t\t\tgraph_chain = g;\n+\t\t\t\tvalid = 1;\n+\t\t\t} else {\n+\t\t\t\tcommit_graph_free(g);\n \t\t\t}\n+\n+\t\t\tbreak;\n \t\t}\n \n \t\tif (!valid) {\n@@ -1377,49 +1387,64 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n \treturn graph_chain;\n }\n \n-static struct commit_graph *load_commit_graph_chain(struct repository *repo,\n-\t\t\t\t\t\t    const char *dir)\n+static int load_commit_graph_chain(struct repository *repo,\n+\t\t\t\t   const char *dir,\n+\t\t\t\t   struct commit_graph **out)\n {\n \tchar *chain_file = get_commit_graph_chain_filename(dir);\n \tstruct stat st;\n \tint fd;\n-\tstruct commit_graph *g = NULL;\n+\tint ret;\n \n-\tif (open_commit_graph_chain(chain_file, &fd, &st, repo->hash_algo)) {\n-\t\t/* ownership of fd is taken over by load function */\n-\t\tg = load_commit_graph_chain_fd_st(repo->objects, fd, &st);\n+\tif (!open_commit_graph_chain(chain_file, &fd, &st, repo->hash_algo)) {\n+\t\tif (errno == ENOENT)\n+\t\t\tret = 1;\n+\t\telse\n+\t\t\tret = error_errno(_(\"could not open commit-graph chain '%s'\"),\n+\t\t\t\t\t  chain_file);\n+\t\tgoto out;\n \t}\n \n+\t/* ownership of fd is taken over by load function */\n+\t*out = load_commit_graph_chain_fd_st(repo->objects, fd, &st);\n+\tret = *out ? 0 : -1;\n+\n+out:\n \tfree(chain_file);\n-\treturn g;\n+\treturn ret;\n }\n \n-static struct commit_graph *read_commit_graph_one(struct repository *repo,\n-\t\t\t\t\t\t  const char *dir)\n+int odb_source_files_read_commit_graph_dir(struct odb_source_files *files,\n+\t\t\t\t\t   struct odb_files_dir *dir,\n+\t\t\t\t\t   struct commit_graph **out)\n {\n-\tstruct commit_graph *g = load_commit_graph_v1(repo, dir);\n+\tstruct repository *repo = files->base.odb->repo;\n+\tint ret;\n \n-\tif (!g)\n-\t\tg = load_commit_graph_chain(repo, dir);\n+\tret = load_commit_graph_v1(repo, dir->abspath, out);\n+\tif (ret > 0)\n+\t\tret = load_commit_graph_chain(repo, dir->abspath, out);\n \n-\treturn g;\n+\treturn ret;\n }\n \n static int odb_source_files_read_commit_graph(struct odb_source *source,\n \t\t\t\t\t      struct commit_graph **out)\n {\n \tstruct odb_source_files *files = odb_source_files_downcast(source);\n+\tbool error = false;\n \n \tfor (struct odb_files_dir *dir = files->dirs; dir; dir = dir->next) {\n-\t\tstruct commit_graph *g = read_commit_graph_one(files->base.odb->repo,\n-\t\t\t\t\t\t\t       dir->abspath);\n-\t\tif (!g)\n+\t\tint ret = odb_source_files_read_commit_graph_dir(files, dir, out);\n+\t\tif (ret) {\n+\t\t\tif (ret < 0)\n+\t\t\t\terror = true;\n \t\t\tcontinue;\n-\t\t*out = g;\n+\t\t}\n \t\treturn 0;\n \t}\n \n-\treturn 1;\n+\treturn error ? -1 : 1;\n }\n \n static void mark_commit_graphs(struct commit_graph *tip,\n@@ -1482,17 +1507,18 @@ static void expire_commit_graphs(const char *dirpath,\n \tstrbuf_release(&path);\n }\n \n-int commit_graph_write(struct repository *r,\n-\t\t       const char *dir,\n-\t\t       const struct commit_graph_write_options *opts)\n+int odb_source_files_write_commit_graph_dir(struct odb_source_files *files,\n+\t\t\t\t\t    struct odb_files_dir *dir,\n+\t\t\t\t\t    const struct commit_graph_write_options *opts)\n {\n+\tstruct repository *r = files->base.odb->repo;\n \tstruct string_list chain_hashes = STRING_LIST_INIT_DUP;\n \tstruct string_list kept = STRING_LIST_INIT_DUP;\n \tstruct tempfile *graph_layer = NULL;\n \tstruct lock_file lk = LOCK_INIT;\n \tstruct commit_graph *g, *chain, *base = NULL;\n \tchar *base_rename_from = NULL, *base_rename_to = NULL;\n-\tchar *graph_name = NULL, *abs_dir = NULL, *new_path = NULL;\n+\tchar *graph_name = NULL, *new_path = NULL;\n \tsize_t mergeable_layers = 0, chainable_layers = 0;\n \tunsigned char hash[GIT_MAX_RAWSZ];\n \tbool mergeable = true;\n@@ -1503,8 +1529,6 @@ int commit_graph_write(struct repository *r,\n \tif (!commit_graph_write_allowed(r))\n \t\treturn 0;\n \n-\tabs_dir = absolute_pathdup(dir);\n-\n \tg = prepare_commit_graph(r);\n \n \t/*\n@@ -1520,7 +1544,7 @@ int commit_graph_write(struct repository *r,\n \tfor (struct commit_graph *layer = g; layer; layer = layer->base_graph) {\n \t\tchar *path = get_commit_graph_filename(layer->dir);\n \t\tbool is_split = !!strcmp(layer->filename, path);\n-\t\tbool is_local = !strcmp(layer->dir, abs_dir);\n+\t\tbool is_local = !strcmp(layer->dir, dir->abspath);\n \n \t\tfree(path);\n \n@@ -1535,9 +1559,9 @@ int commit_graph_write(struct repository *r,\n \t}\n \n \tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tgraph_name = xstrfmt(\"%s/info/commit-graphs/tmp_graph_XXXXXX\", abs_dir);\n+\t\tgraph_name = xstrfmt(\"%s/info/commit-graphs/tmp_graph_XXXXXX\", dir->abspath);\n \t} else {\n-\t\tgraph_name = get_commit_graph_filename(abs_dir);\n+\t\tgraph_name = get_commit_graph_filename(dir->abspath);\n \t}\n \n \tif (safe_create_leading_directories(r, graph_name)) {\n@@ -1547,7 +1571,7 @@ int commit_graph_write(struct repository *r,\n \t}\n \n \tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n-\t\tchar *chain_name = get_commit_graph_chain_filename(abs_dir);\n+\t\tchar *chain_name = get_commit_graph_chain_filename(dir->abspath);\n \n \t\trepo_hold_lock_file_for_update_mode(r, &lk, chain_name,\n \t\t\t\t\t\t    LOCK_DIE_ON_ERROR, 0444);\n@@ -1586,7 +1610,7 @@ int commit_graph_write(struct repository *r,\n \tif (opts->flags & COMMIT_GRAPH_WRITE_SPLIT) {\n \t\tconst char *new_hash = hash_to_hex_algop(hash, r->hash_algo);\n \n-\t\tnew_path = get_split_graph_filename(abs_dir, new_hash);\n+\t\tnew_path = get_split_graph_filename(dir->abspath, new_hash);\n \n \t\tstring_list_append(&chain_hashes, new_hash);\n \t\tfor (chain = base; chain; chain = chain->base_graph)\n@@ -1632,7 +1656,7 @@ int commit_graph_write(struct repository *r,\n \t\t\t\tgoto out;\n \t\t\t}\n \t\t} else if (!base) {\n-\t\t\tchar *graph_name = get_commit_graph_filename(abs_dir);\n+\t\t\tchar *graph_name = get_commit_graph_filename(dir->abspath);\n \t\t\tunlink(graph_name);\n \t\t\tfree(graph_name);\n \t\t}\n@@ -1653,12 +1677,12 @@ int commit_graph_write(struct repository *r,\n \tcommit_lock_file(&lk);\n \n \tif (!(opts->flags & COMMIT_GRAPH_WRITE_SPLIT)) {\n-\t\tchar *chain_name = get_commit_graph_chain_filename(abs_dir);\n+\t\tchar *chain_name = get_commit_graph_chain_filename(dir->abspath);\n \t\tunlink(chain_name);\n \t\tfree(chain_name);\n \t}\n \n-\texpire_commit_graphs(abs_dir, &kept,\n+\texpire_commit_graphs(dir->abspath, &kept,\n \t\t\t     opts->expire_time ? opts->expire_time : (timestamp_t) time(NULL));\n \n \tret = 0;\n@@ -1672,7 +1696,6 @@ int commit_graph_write(struct repository *r,\n \tfree(base_rename_to);\n \tfree(graph_name);\n \tfree(new_path);\n-\tfree(abs_dir);\n \treturn ret;\n }\n \n@@ -1680,7 +1703,7 @@ static int odb_source_files_write_commit_graph(struct odb_source *source,\n \t\t\t\t\t       const struct commit_graph_write_options *opts)\n {\n \tstruct odb_source_files *files = odb_source_files_downcast(source);\n-\treturn commit_graph_write(source->odb->repo, files->dirs->abspath, opts);\n+\treturn odb_source_files_write_commit_graph_dir(files, files->dirs, opts);\n }\n \n static int fsck_commit_graph(struct odb_source_files *files,\ndiff --git a/odb/source-files.h b/odb/source-files.h\nindex f779e67a1db..71ccacd8d90 100644\n--- a/odb/source-files.h\n+++ b/odb/source-files.h\n@@ -106,4 +106,16 @@ static inline struct odb_source_files *odb_source_files_downcast(struct odb_sour\n  */\n struct odb_files_dir *odb_source_files_find_dir(struct object_database *odb, const char *obj_dir);\n \n+/*\n+ * Read and write a commit graph via a specific object directory. These\n+ * functions should not be used, but we have to provide this ability to support\n+ * the \"--object-dir=\" option for git-commit-graph(1).\n+ */\n+int odb_source_files_read_commit_graph_dir(struct odb_source_files *files,\n+\t\t\t\t\t   struct odb_files_dir *dir,\n+\t\t\t\t\t   struct commit_graph **out);\n+int odb_source_files_write_commit_graph_dir(struct odb_source_files *files,\n+\t\t\t\t\t    struct odb_files_dir *dir,\n+\t\t\t\t\t    const struct commit_graph_write_options *opts);\n+\n #endif\n\n-- \n2.56.1.763.g16177fe143.dirty\n\n\n"},{"id":"558575","messageId":"xmqqqzgkjy57.fsf@gitster.g","threadId":"66855","inReplyTo":"20261116-pks-odb-pluggable-commit-graphs-v1-1-a83fc1ff0808@pks.im","subject":"Re: [PATCH 01/12] commit-graph: simplify linking commit graph chains","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-11-16T20:53:56Z","receivedAt":"2026-11-16T20:53:56Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> When we load a commit graph chain we verify for every layer that the\n> hash of the base commit graph matches what the current tip has in its\n> base graphs chunk. To do so we pass both the chain itself as well as the\n> array of object IDs that we have read from the commit-graph chain file\n> to `add_graph_to_chain()`.\n>\n> But the array of object IDs is redundant information: we have already\n> loaded and linked the base layers, and as part of that we have stored\n> their respective hashes in the structure itself. Consequently, we can\n> already derive the hashes by walking the chain of base layers.\n>\n> Refactor the function to only take the new tip and base layers as input\n> and derive all the other information ad-hoc and make the function\n> public. This prepares for a subsequent commit, where we are about to\n> move the logic that creates commit graph chains into the object database\n> backend.\n>\n> Signed-off-by: Patrick Steinhardt <ps@pks.im>\n> ---\n>  commit-graph.c | 52 ++++++++++++++++++++++++----------------------------\n>  commit-graph.h |  8 ++++++++\n>  2 files changed, 32 insertions(+), 28 deletions(-)\n>\n> diff --git a/commit-graph.c b/commit-graph.c\n> index da65cd3deb9..94c761907a0 100644\n> --- a/commit-graph.c\n> +++ b/commit-graph.c\n> @@ -575,50 +575,48 @@ static void validate_mixed_bloom_settings(struct commit_graph *g)\n>  \t}\n>  }\n>  \n> -static int add_graph_to_chain(struct commit_graph *g,\n> -\t\t\t      struct commit_graph *chain,\n> -\t\t\t      struct object_id *oids,\n> -\t\t\t      int n)\n> +int commit_graph_link_base(struct commit_graph *g,\n> +\t\t\t   struct commit_graph *base)\n\nNice simplification, I guess.\n\n>  {\n> -\tstruct commit_graph *cur_g = chain;\n> +\tstruct commit_graph *layer;\n> +\tsize_t n = 0;\n> +\n> +\tfor (layer = base; layer; layer = layer->base_graph)\n> +\t\tn++;\n\nWe chase the pointers to count the number of layers.  We no longer\ncount in signed int, which is probably a big plus in the sanitation\ndepartment.\n\n>  \tif (n && !g->chunk_base_graphs) {\n>  \t\twarning(_(\"commit-graph has no base graphs chunk\"));\n> -\t\treturn 0;\n> +\t\treturn -1;\n>  \t}\n\nIt might be obvious when reading (and writing) this patch, but it is\na welcome change to standardize on \"0 means happy\" convention.\nShould the fact that we now use more common error reporting\nconvention be mentioned in the log message?\n\n>  \tif (g->chunk_base_graphs_size / g->hash_algo->rawsz < n) {\n>  \t\twarning(_(\"commit-graph base graphs chunk is too small\"));\n> -\t\treturn 0;\n> +\t\treturn -1;\n>  \t}\n>  \n> -\twhile (n) {\n> +\tfor (layer = base; layer; layer = layer->base_graph) {\n>  \t\tn--;\n> -\n> -\t\tif (!cur_g ||\n> -\t\t    !oideq(&oids[n], &cur_g->oid) ||\n> -\t\t    !hasheq(oids[n].hash, g->chunk_base_graphs + st_mult(g->hash_algo->rawsz, n),\n> +\t\tif (!hasheq(layer->oid.hash,\n> +\t\t\t    g->chunk_base_graphs + st_mult(g->hash_algo->rawsz, n),\n>  \t\t\t    g->hash_algo)) {\n\nWe no longer check cur_g->oid is what we expect, because we lost the\narray that holds the value to compare.  Would this make cur_g->oid a\nwrite-only member that is only computed and written but never used\nother than reporting purposes?  In commit-graph.c before this step,\nI think this is the only place that reads ->oid for purposes other\nthan passing it to diagnostics like die/warn.\n\nWe seem to still be making sure in load_commit_graph_chain_fd_st()\nthat the line we read into local oid there is a hash, so perhaps it\nis possible to validate g->oid against this oid after we load g?\n\n> @@ -682,7 +679,7 @@ struct commit_graph *load_commit_graph_chain_fd_st(struct object_database *odb,\n>  \t\t\tfree(graph_name);\n>  \n>  \t\t\tif (g) {\n> -\t\t\t\tif (add_graph_to_chain(g, graph_chain, oids, i)) {\n> +\t\t\t\tif (!commit_graph_link_base(g, graph_chain)) {\n>  \t\t\t\t\tgraph_chain = g;\n>  \t\t\t\t\tvalid = 1;\n>  \t\t\t\t} else {\n\n> +\n> +/*\n> + * Link the commit graph layer `g` on top of the chain `base`. `base` may be\n> + * NULL if `g` is the root of a new chain. Returns 0 on success, a negative\n> + * error code otherwise.\n> + */\n> +int commit_graph_link_base(struct commit_graph *g,\n> +\t\t\t   struct commit_graph *base);\n>  struct commit_graph *read_commit_graph_one(struct repository *repo,\n>  \t\t\t\t\t   const char *dir);\n\n"}]}