{"thread":{"id":"65846","subject":"[PATCH/RFC 0/6] commit-reach: terminate merge-base walk when one side is exhausted","startedAt":"2026-06-20T10:37:03Z","lastAt":"2026-08-12T20:05:24Z","messageCount":176,"participants":["Kristofer Karlsson via GitGitGadget","Elijah Newren via GitGitGadget","Derrick Stolee","Kristofer Karlsson","Junio C Hamano","René Scharfe","SZEDER Gábor","Elijah Newren"],"isPatch":true,"patchVersion":1,"patchTotal":6},"messages":[{"id":"546021","messageId":"pull.2149.git.1781951820.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":null,"subject":"[PATCH/RFC 0/6] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-20T10:36:53Z","receivedAt":"2026-06-20T10:37:03Z","isPatch":true,"body":"Hi,\n\nThis follows up on my RFC [1] with a concrete proposal. I expect the design\nto still be scrutinized, but that may be easier with actual code to look at.\n\nI tried to make this easier to review by splitting into atomic patches. The\nfirst two patches are the meatiest parts, though they are pure refactoring.\nThe behavior change is in patch 3 and is in itself quite small. The last\npatch adds technical documentation to support future development.\n\n----------------------------------------------------------------------------\n\nOptimize paint_down_to_common() for merge-base queries that hit large\none-sided histories.\n\nWhen the walk from one side reaches a commit with a very low generation\nnumber that the other side never paints, the walk is forced to drain most of\nthe graph. A common trigger is a repository import that grafts a separate\nhistory with its own root, but any merge that introduces a low-generation\ncommit never painted by the other side has the same effect.\n\nA new merge-base candidate can only be discovered when exclusive PARENT1 and\nPARENT2 paint meet. This series teaches paint_down_to_common() to stop as\nsoon as one side has no exclusive commits left in the queue; once one side\nis exhausted, no further candidates can appear.\n\n origin/HEAD o o PR HEAD\n | |\n (import) o :\n / \\ /\n | o merge-base\n | |\n : : (~2.5M commits)\n | |\n import root main root\n\n\nIn the RFC thread [1], Derrick Stolee provided a criss-cross counterexample\nthat sharpened the halt condition, and Elijah Newren independently\ndiscovered the same optimization and shared an implementation in PR #2150\n[2]. Patch 4 incorporates test cases from Elijah's branch.\n\nThis series implements the optimization only after the walk enters the\nfinite-generation region, where generation ordering guarantees that paint on\nvisited commits is final.\n\nPatch layout:\n\n1/6 commit-reach: decouple ahead_behind from nonstale_queue 2/6\ncommit-reach: introduce paint_queue and per-side counters 3/6 commit-reach:\nstop the walk when one side is exhausted 4/6 t6600: add side-exhaustion\nedge-case tests 5/6 t6099, t6600: add side-exhaustion regression tests 6/6\nDocumentation/technical: document paint_down_to_common()\n\nBenchmarks\n\nMeasured on a 2.6M-commit monorepo with commit-graph (baseline v2.55-rc1):\n\nmerge-base --all (across import) 4.293s -> 8ms (537x)\nmerge-tree (across import) 5.345s -> 13ms (411x)\nmerge-base --all (1000 commits apart) 5.404s -> 7ms (772x)\n\n\nNo regression on linux.git (1.4M commits, commit-graph):\n\nmerge-base HEAD HEAD~1000 38ms -> 40ms\nmerge-base --all HEAD HEAD~1000 87ms -> 36ms\nmerge-base --is-ancestor HEAD~1000 HEAD 11ms -> 11ms\nmerge-base --all HEAD HEAD~10000 626ms -> 428ms\n\n\n[1]\nhttps://lore.kernel.org/git/CAL71e4Ps-2_0+uuZu43N9pFnXBemoAohPs_eyRJf8taXHJPAXQ@mail.gmail.com/T/#u\n[2] https://github.com/gitgitgadget/git/pull/2150\n\nElijah Newren (1):\n t6600: add test cases for side-exhaustion edge cases\n\nKristofer Karlsson (5):\n commit-reach: decouple ahead_behind from nonstale_queue\n commit-reach: introduce struct paint_queue with per-side counters\n commit-reach: terminate merge-base walk when one paint side is\n exhausted\n t6099, t6600: add side-exhaustion regression tests\n Documentation/technical: add paint-down-to-common doc\n\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 130 ++++++++++++++\n commit-reach.c | 159 +++++++++++-------\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 +++++++++\n t/t6600-test-reach.sh | 136 +++++++++++++++\n 7 files changed, 451 insertions(+), 59 deletions(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\n\nbase-commit: 8d96f09e9245ddf80c1981476fcbac8c4bb4125f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2149%2Fspkrka%2Fside-exhaust-pr-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2149/spkrka/side-exhaust-pr-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/2149\n-- \ngitgitgadget\n"},{"id":"546022","messageId":"5492acda0ad05eab67198880a5262e84a3f22ba6.1781951820.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.git.1781951820.gitgitgadget@gmail.com","subject":"[PATCH/RFC 1/6] commit-reach: decouple ahead_behind from nonstale_queue","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-20T10:36:54Z","receivedAt":"2026-06-20T10:37:04Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nMove ahead_behind() off the shared nonstale_queue abstraction to use\na plain prio_queue with a local max_nonstale pointer. The nonstale\ntracking is inlined into insert_no_dup().\n\nThis prepares for replacing nonstale_queue with a paint_queue struct\nthat tracks per-side commit counts, which ahead_behind() does not\nneed. No behavior change.\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 32 +++++++++++++++++++++-----------\n 1 file changed, 21 insertions(+), 11 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 5df471a313..377a5cc42a 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -1089,12 +1089,18 @@ struct commit_list *get_reachable_subset(struct commit **from, size_t nr_from,\n define_commit_slab(bit_arrays, struct bitmap *);\n static struct bit_arrays bit_arrays;\n \n-static void insert_no_dup(struct nonstale_queue *queue, struct commit *c)\n+static void insert_no_dup(struct prio_queue *queue,\n+\t\t\t struct commit **max_nonstale,\n+\t\t\t struct commit *c)\n {\n \tif (c->object.flags & PARENT2)\n \t\treturn;\n-\tnonstale_queue_put(queue, c);\n \tc->object.flags |= PARENT2;\n+\tprio_queue_put(queue, c);\n+\tif (!(c->object.flags & STALE) &&\n+\t (!*max_nonstale ||\n+\t queue->compare(*max_nonstale, c, queue->cb_data) <= 0))\n+\t\t*max_nonstale = c;\n }\n \n static struct bitmap *get_bit_array(struct commit *c, int width)\n@@ -1118,9 +1124,10 @@ void ahead_behind(struct repository *r,\n \t\t struct commit **commits, size_t commits_nr,\n \t\t struct ahead_behind_count *counts, size_t counts_nr)\n {\n-\tstruct nonstale_queue queue = {\n-\t\t{ .compare = compare_commits_by_gen_then_commit_date }\n+\tstruct prio_queue queue = {\n+\t\t.compare = compare_commits_by_gen_then_commit_date\n \t};\n+\tstruct commit *max_nonstale = NULL;\n \tsize_t width = DIV_ROUND_UP(commits_nr, BITS_IN_EWORD);\n \n \tif (!commits_nr || !counts_nr)\n@@ -1140,14 +1147,17 @@ void ahead_behind(struct repository *r,\n \t\tstruct bitmap *bitmap = get_bit_array(c, width);\n \n \t\tbitmap_set(bitmap, i);\n-\t\tinsert_no_dup(&queue, c);\n+\t\tinsert_no_dup(&queue, &max_nonstale, c);\n \t}\n \n-\twhile (queue.max_nonstale) {\n-\t\tstruct commit *c = nonstale_queue_get(&queue);\n+\twhile (max_nonstale) {\n+\t\tstruct commit *c = prio_queue_get(&queue);\n \t\tstruct commit_list *p;\n \t\tstruct bitmap *bitmap_c = get_bit_array(c, width);\n \n+\t\tif (c == max_nonstale)\n+\t\t\tmax_nonstale = NULL;\n+\n \t\tfor (size_t i = 0; i < counts_nr; i++) {\n \t\t\tint reach_from_tip = !!bitmap_get(bitmap_c, counts[i].tip_index);\n \t\t\tint reach_from_base = !!bitmap_get(bitmap_c, counts[i].base_index);\n@@ -1178,7 +1188,7 @@ void ahead_behind(struct repository *r,\n \t\t\tif (bitmap_popcount(bitmap_p) == commits_nr)\n \t\t\t\tp->item->object.flags |= STALE;\n \n-\t\t\tinsert_no_dup(&queue, p->item);\n+\t\t\tinsert_no_dup(&queue, &max_nonstale, p->item);\n \t\t}\n \n \t\tfree_bit_array(c);\n@@ -1186,10 +1196,10 @@ void ahead_behind(struct repository *r,\n \n \t/* STALE is used here, PARENT2 is used by insert_no_dup(). */\n \trepo_clear_commit_marks(r, PARENT2 | STALE);\n-\tfor (size_t i = 0; i < queue.pq.nr; i++)\n-\t\tfree_bit_array(queue.pq.array[i].data);\n+\tfor (size_t i = 0; i < queue.nr; i++)\n+\t\tfree_bit_array(queue.array[i].data);\n \tclear_bit_arrays(&bit_arrays);\n-\tclear_nonstale_queue(&queue);\n+\tclear_prio_queue(&queue);\n }\n \n struct commit_and_index {\n-- \ngitgitgadget\n\n"},{"id":"546023","messageId":"316e4dfe261043730c77142639f86f5c3cabe370.1781951820.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.git.1781951820.gitgitgadget@gmail.com","subject":"[PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-20T10:36:55Z","receivedAt":"2026-06-20T10:37:06Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nReplace the nonstale_queue abstraction in paint_down_to_common() with\na new paint_queue struct that tracks per-side commit counts. Each\nnon-stale queued commit occupies exactly one counter bucket based on\nits paint flags: PARENT1-only, PARENT2-only, or both sides (a pending\nmerge-base candidate).\n\nThe counters are maintained by paint_count_transition() which handles\nall flag changes as bucket transfers: remove from the old bucket, add\nto the new one. Either step is a no-op when the respective state has\nno bucket (stale or zero).\n\nThe loop now drains the queue via paint_queue_get() and breaks when\nall counters reach zero, replacing the old pointer-based termination\n(max_nonstale). This is equivalent behavior.\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 114 ++++++++++++++++++++++++++++---------------------\n 1 file changed, 66 insertions(+), 48 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 377a5cc42a..ba1e896f0f 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -41,58 +41,75 @@ static int compare_commits_by_gen(const void *_a, const void *_b)\n }\n \n /*\n- * A prio_queue with O(1) termination check. 'max_nonstale' tracks\n- * the lowest-priority non-stale commit enqueued so far; once it is\n- * popped, every remaining entry is known to be STALE.\n+ * Priority queue with per-side commit counters for paint_down_to_common().\n+ * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n+ * PARENT2-only, or both (a pending merge-base candidate).\n */\n-struct nonstale_queue {\n+struct paint_queue {\n \tstruct prio_queue pq;\n-\tstruct commit *max_nonstale;\n+\tint p1_count;\n+\tint p2_count;\n+\tint pending_merge_bases;\n };\n \n-static void nonstale_queue_put(struct nonstale_queue *queue,\n-\t\t\t struct commit *c)\n+/*\n+ * Adjust per-side counters for a paint-state transition. Non-stale\n+ * commits are counted in one of three counters: PARENT1-only,\n+ * PARENT2-only, or both. Zero means \"not in the queue\" (used on\n+ * enqueue/dequeue); stale commits are not counted at all.\n+ */\n+static void paint_count_transition(struct paint_queue *queue,\n+\t\t\t\t unsigned old_flags, unsigned new_flags)\n {\n-\tstruct commit *old = queue->max_nonstale;\n+\tunsigned old_paint = old_flags & (PARENT1 | PARENT2 | STALE);\n+\tunsigned new_paint = new_flags & (PARENT1 | PARENT2 | STALE);\n \n-\tprio_queue_put(&queue->pq, c);\n-\tif (c->object.flags & STALE)\n+\tif (old_paint == new_paint)\n \t\treturn;\n-\tif (!old || queue->pq.compare(old, c, queue->pq.cb_data) <= 0)\n-\t\tqueue->max_nonstale = c;\n-}\n-\n-static struct commit *nonstale_queue_get(struct nonstale_queue *queue)\n-{\n-\tstruct commit *commit = prio_queue_get(&queue->pq);\n \n-\tif (commit == queue->max_nonstale)\n-\t\tqueue->max_nonstale = NULL;\n-\n-\treturn commit;\n+\tif (!(old_paint & STALE)) {\n+\t\tswitch (old_paint & (PARENT1 | PARENT2)) {\n+\t\tcase 0: break;\n+\t\tcase PARENT1: queue->p1_count--; break;\n+\t\tcase PARENT2: queue->p2_count--; break;\n+\t\tcase PARENT1 | PARENT2: queue->pending_merge_bases--; break;\n+\t\tdefault: BUG(\"unexpected paint state\");\n+\t\t}\n+\t}\n+\tif (!(new_paint & STALE)) {\n+\t\tswitch (new_paint & (PARENT1 | PARENT2)) {\n+\t\tcase 0: break;\n+\t\tcase PARENT1: queue->p1_count++; break;\n+\t\tcase PARENT2: queue->p2_count++; break;\n+\t\tcase PARENT1 | PARENT2: queue->pending_merge_bases++; break;\n+\t\tdefault: BUG(\"unexpected paint state\");\n+\t\t}\n+\t}\n }\n \n-static void clear_nonstale_queue(struct nonstale_queue *queue)\n+static void paint_queue_put(struct paint_queue *queue,\n+\t\t\t struct commit *c, unsigned add_flags)\n {\n-\tclear_prio_queue(&queue->pq);\n-\tqueue->max_nonstale = NULL;\n-}\n+\tunsigned old_flags = c->object.flags;\n+\tc->object.flags |= add_flags;\n \n-static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n-\t\t\t\t struct commit *c)\n-{\n-\tif (c->object.flags & ENQUEUED)\n-\t\treturn;\n-\tc->object.flags |= ENQUEUED;\n-\tnonstale_queue_put(queue, c);\n+\tif (old_flags & ENQUEUED) {\n+\t\tpaint_count_transition(queue, old_flags, c->object.flags);\n+\t} else {\n+\t\tc->object.flags |= ENQUEUED;\n+\t\tprio_queue_put(&queue->pq, c);\n+\t\tpaint_count_transition(queue, 0, c->object.flags);\n+\t}\n }\n \n-static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n+static struct commit *paint_queue_get(struct paint_queue *queue)\n {\n-\tstruct commit *commit = nonstale_queue_get(queue);\n+\tstruct commit *commit = prio_queue_get(&queue->pq);\n \n-\tif (commit)\n+\tif (commit) {\n \t\tcommit->object.flags &= ~ENQUEUED;\n+\t\tpaint_count_transition(queue, commit->object.flags, 0);\n+\t}\n \treturn commit;\n }\n \n@@ -104,9 +121,10 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\tenum merge_base_flags mb_flags,\n \t\t\t\tstruct commit_list **result)\n {\n-\tstruct nonstale_queue queue = {\n-\t\t{ compare_commits_by_gen_then_commit_date }\n+\tstruct paint_queue queue = {\n+\t\t.pq = { compare_commits_by_gen_then_commit_date }\n \t};\n+\tstruct commit *commit;\n \tint i;\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n@@ -119,15 +137,12 @@ static int paint_down_to_common(struct repository *r,\n \t\tcommit_list_append(one, result);\n \t\treturn 0;\n \t}\n-\tnonstale_queue_put_dedup(&queue, one);\n+\tpaint_queue_put(&queue, one, 0);\n \n-\tfor (i = 0; i < n; i++) {\n-\t\ttwos[i]->object.flags |= PARENT2;\n-\t\tnonstale_queue_put_dedup(&queue, twos[i]);\n-\t}\n+\tfor (i = 0; i < n; i++)\n+\t\tpaint_queue_put(&queue, twos[i], PARENT2);\n \n-\twhile (queue.max_nonstale) {\n-\t\tstruct commit *commit = nonstale_queue_get_dedup(&queue);\n+\twhile ((commit = paint_queue_get(&queue))) {\n \t\tstruct commit_list *parents;\n \t\tint flags;\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n@@ -165,7 +180,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t\tif ((p->object.flags & flags) == flags)\n \t\t\t\tcontinue;\n \t\t\tif (repo_parse_commit(r, p)) {\n-\t\t\t\tclear_nonstale_queue(&queue);\n+\t\t\t\tclear_prio_queue(&queue.pq);\n \t\t\t\tcommit_list_free(*result);\n \t\t\t\t*result = NULL;\n \t\t\t\t/*\n@@ -180,12 +195,15 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\treturn error(_(\"could not parse commit %s\"),\n \t\t\t\t\t oid_to_hex(&p->object.oid));\n \t\t\t}\n-\t\t\tp->object.flags |= flags;\n-\t\t\tnonstale_queue_put_dedup(&queue, p);\n+\t\t\tpaint_queue_put(&queue, p, flags);\n \t\t}\n+\n+\t\tif (queue.p1_count + queue.p2_count +\n+\t\t queue.pending_merge_bases == 0)\n+\t\t\tbreak;\n \t}\n \n-\tclear_nonstale_queue(&queue);\n+\tclear_prio_queue(&queue.pq);\n \tcommit_list_sort_by_date(result);\n \treturn 0;\n }\n-- \ngitgitgadget\n\n"},{"id":"546024","messageId":"ed12a5cb5b76925cff08d2ab61efeda382b4477a.1781951820.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.git.1781951820.gitgitgadget@gmail.com","subject":"[PATCH/RFC 3/6] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-20T10:36:56Z","receivedAt":"2026-06-20T10:37:09Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd an early termination check to paint_down_to_common() using the\nper-side counters introduced in the previous commit. Once the walk\nenters the finite-generation region, terminate early when one side's\nexclusive count drops to zero -- no new merge-base can form without\nboth paint sides meeting.\n\nThe check also waits for pending_merge_bases to reach zero, ensuring\nall merge-base candidates have been popped and recorded before\nexiting.\n\nThe INFINITY gate ensures correctness: commits without a commit-graph\nentry have GENERATION_NUMBER_INFINITY and are ordered by commit date,\nwhich is not topologically reliable. The optimization only fires\nonce the walk enters the finite-generation region where ordering\nguarantees hold.\n\nOn large repositories with commit-graph, this yields 100-1000x\nspeedups for merge-base queries where one side (e.g. a PR branch) is\nmuch smaller than the other.\n\nHelped-by: Derrick Stolee \nHelped-by: Elijah Newren \nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 13 +++++++++++++\n 1 file changed, 13 insertions(+)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex ba1e896f0f..fcd1ad0167 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -201,6 +201,19 @@ static int paint_down_to_common(struct repository *r,\n \t\tif (queue.p1_count + queue.p2_count +\n \t\t queue.pending_merge_bases == 0)\n \t\t\tbreak;\n+\n+\t\t/*\n+\t\t * Side exhaustion: a new merge-base can only form\n+\t\t * when both PARENT1-only and PARENT2-only commits\n+\t\t * remain in the queue. In the finite-generation\n+\t\t * region the queue is ordered topologically, so\n+\t\t * no future step can add paint to visited commits\n+\t\t * and an exhausted side cannot reappear.\n+\t\t */\n+\t\tif (generation < GENERATION_NUMBER_INFINITY &&\n+\t\t queue.pending_merge_bases == 0 &&\n+\t\t (queue.p1_count == 0 || queue.p2_count == 0))\n+\t\t\tbreak;\n \t}\n \n \tclear_prio_queue(&queue.pq);\n-- \ngitgitgadget\n\n"},{"id":"546025","messageId":"91372b975fbe102538c05c7d2cdae356539d1bbd.1781951820.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.git.1781951820.gitgitgadget@gmail.com","subject":"[PATCH/RFC 4/6] t6600: add test cases for side-exhaustion edge cases","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-20T10:36:57Z","receivedAt":"2026-06-20T10:37:11Z","isPatch":true,"body":"From: Elijah Newren \n\nAdd test cases to t6600-test-reach.sh that exercise edge cases in the\nside-exhaustion optimization for paint_down_to_common():\n\n - in_merge_bases_many:self: commit is both A and one of the X inputs\n - get_merge_bases_many:duplicate-twos: duplicate entries in X list\n - get_merge_bases_many:pending-stale: STALE transition on an\n already-painted commit (ps-* diamond topology)\n - get_merge_bases_many:infinity-both-sides: both tips outside the\n commit-graph with non-monotonic dates (pi-* topology)\n\nSigned-off-by: Elijah Newren \nSigned-off-by: Kristofer Karlsson \n---\n t/t6600-test-reach.sh | 111 ++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 111 insertions(+)\n\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex b5b314e570..775c077c87 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -49,6 +49,62 @@ test_expect_success 'setup' '\n \t\t\tgit tag -a -m \"$x-$i\" tag-$x-$i commit-$x-$i || return 1\n \t\tdone\n \tdone &&\n+\n+\t# Build a small side topology to exercise the (PARENT1|PARENT2) ->\n+\t# (PARENT1|PARENT2|STALE) transition in paint_down_to_common(); the\n+\t# 10x10 grid above does not exercise it because no merge-base candidate\n+\t# there is a descendant of another, so STALE never reaches a\n+\t# still-pending candidate.\n+\t#\n+\t# ps-X\n+\t# /|\\\n+\t# / | \\\n+\t# ps-Z ps-B ps-W\n+\t# | / \\ |\n+\t# | / \\ |\n+\t# |/ \\|\n+\t# ps-T1 ps-T2\n+\t#\n+\t# where ps-T1=merge(ps-Z,ps-B), ps-T2=merge(ps-W,ps-B), so\n+\t# merge-base(ps-T1,ps-T2) = ps-B. During the walk, ps-X transitions\n+\t# to (PARENT1|PARENT2) via ps-Z and ps-W before ps-B is dequeued;\n+\t# then the STALE-walk from ps-B transitions ps-X to\n+\t# (PARENT1|PARENT2|STALE).\n+\tgit checkout --orphan ps-orphan &&\n+\ttest_commit ps-X &&\n+\tgit checkout -b ps-B-br ps-X && test_commit ps-B &&\n+\tgit checkout -b ps-Z-br ps-X && test_commit ps-Z &&\n+\tgit checkout -b ps-W-br ps-X && test_commit ps-W &&\n+\tgit checkout -b ps-T1 ps-Z &&\n+\tgit merge --no-ff -m ps-T1 ps-B &&\n+\tgit checkout -b ps-T2 ps-W &&\n+\tgit merge --no-ff -m ps-T2 ps-B &&\n+\n+\t# Build a side topology that lives entirely outside the half\n+\t# commit-graph and has non-monotonic commit dates, to exercise the\n+\t# INFINITY-gate in paint_down_to_common. With both tips outside\n+\t# the graph, generation is INFINITY and the queue falls back to\n+\t# commit-date order, which here is non-monotonic.\n+\t#\n+\t# pi-X (date 500, PARENT1 tip) --> pi-P, pi-D\n+\t# pi-D (date 480) --> pi-C\n+\t# pi-C (date 200) --> pi-B\n+\t# pi-B (date 100, PARENT2 tip) --> pi-P\n+\t# pi-P (date 450, root)\n+\t#\n+\t# merge-base(pi-X, pi-B) = pi-B (it is an ancestor of pi-X and is\n+\t# itself one of the queried tips).\n+\tgit checkout --orphan pi-orphan &&\n+\ttest_commit --date \"@450 +0000\" pi-P &&\n+\ttest_commit --date \"@100 +0000\" pi-B &&\n+\ttest_commit --date \"@200 +0000\" pi-C &&\n+\ttest_commit --date \"@480 +0000\" pi-D &&\n+\tGIT_AUTHOR_DATE=\"@500 +0000\" GIT_COMMITTER_DATE=\"@500 +0000\" \\\n+\t\tgit commit-tree -p pi-D -p pi-P -m pi-X pi-D^{tree} >pi-X-oid &&\n+\tpi_x=\"$(cat pi-X-oid)\" &&\n+\tgit branch -f pi-X-br \"$pi_x\" &&\n+\tgit tag pi-X \"$pi_x\" &&\n+\n \tgit commit-graph write --reachable &&\n \tmv .git/objects/info/commit-graph commit-graph-full &&\n \tchmod u+w commit-graph-full &&\n@@ -146,6 +202,16 @@ test_expect_success 'in_merge_bases_many:miss-heuristic' '\n \ttest_all_modes in_merge_bases_many\n '\n \n+test_expect_success 'in_merge_bases_many:self' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-8\n+\tX:commit-5-9\n+\tX:commit-6-8\n+\tEOF\n+\techo \"in_merge_bases_many(A,X):1\" >expect &&\n+\ttest_all_modes in_merge_bases_many\n+'\n+\n test_expect_success 'is_descendant_of:hit' '\n \tcat >input <<-\\EOF &&\n \tA:commit-5-7\n@@ -183,6 +249,51 @@ test_expect_success 'get_merge_bases_many' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'get_merge_bases_many:duplicate-twos' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tX:commit-4-8\n+\tX:commit-4-8\n+\tX:commit-6-6\n+\tX:commit-6-6\n+\tX:commit-8-3\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse commit-5-6 \\\n+\t\t\t commit-4-7 | sort\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n+test_expect_success 'get_merge_bases_many:pending-stale' '\n+\t# Exercises the (PARENT1|PARENT2) -> (...|STALE) transition path in\n+\t# paint_down_to_common(). See the topology comment in the setup test.\n+\tcat >input <<-\\EOF &&\n+\tA:ps-T1\n+\tX:ps-T2\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse ps-B\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n+test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n+\t# Exercises the push-time INFINITY-gate in paint_down_to_common(). See\n+\t# the pi-* topology comment in the setup test.\n+\tcat >input <<-\\EOF &&\n+\tA:pi-X\n+\tX:pi-B\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse pi-B\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546026","messageId":"faf5bc98ede79965e23bfe1535127d6f52221680.1781951820.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.git.1781951820.gitgitgadget@gmail.com","subject":"[PATCH/RFC 5/6] t6099, t6600: add side-exhaustion regression tests","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-20T10:36:58Z","receivedAt":"2026-06-20T10:37:13Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd t6099 to test the case where multiple merge-base candidates exist\nand one is an ancestor of another. This exercises the side-exhaustion\noptimization in paint_down_to_common together with the\nremove_redundant safety net in get_merge_bases_many_0.\n\nAdd a mixed finite/INFINITY test to t6600 where one tip is outside\nthe commit-graph (INFINITY generation) and the other is inside.\nThis exercises the region transition: the walk starts in the\nINFINITY region where side-exhaustion is disabled, then crosses\ninto the finite region where it can fire.\n\nSigned-off-by: Kristofer Karlsson \n---\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 +++++++++++++++++++++++++++\n t/t6600-test-reach.sh | 25 ++++++++\n 3 files changed, 108 insertions(+)\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\ndiff --git a/t/meson.build b/t/meson.build\nindex 3219264fe7..ee6ebdffb9 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -786,6 +786,7 @@ integration_tests = [\n 't6041-bisect-submodule.sh',\n 't6050-replace.sh',\n 't6060-merge-index.sh',\n+ 't6099-merge-base-side-exhaustion.sh',\n 't6100-rev-list-in-order.sh',\n 't6101-rev-parse-parents.sh',\n 't6102-rev-list-unexpected-objects.sh',\ndiff --git a/t/t6099-merge-base-side-exhaustion.sh b/t/t6099-merge-base-side-exhaustion.sh\nnew file mode 100755\nindex 0000000000..bae3ea7f83\n--- /dev/null\n+++ b/t/t6099-merge-base-side-exhaustion.sh\n@@ -0,0 +1,82 @@\n+#!/bin/sh\n+\n+test_description='merge-base with ancestor among merge-base candidates\n+\n+Test that merge-base --all correctly handles cases where\n+multiple merge-base candidates exist and one is an ancestor\n+of another. The side-exhaustion optimization in\n+paint_down_to_common may exit before STALE propagation\n+removes the ancestor, but remove_redundant catches it.\n+\n+Graph shape (parents are below children):\n+\n+ A ----------- X\n+ |\\ /|\n+ | B---------/ |\n+ | | |\n+ e2 \\ f2\n+ | | |\n+ e1 d1 f1\n+ \\ | /\n+ \\ | /\n+ \\| /\n+ C\n+\n+A and X are the two tips.\n+B and C are both reachable from A and X.\n+B reaches C through d1.\n+Only B should appear in merge-base --all output.\n+'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+TEST_PASSES_SANITIZE_LEAK=true\n+. ./test-lib.sh\n+\n+test_expect_success 'setup ancestor merge-base candidate' '\n+\ttest_commit C &&\n+\n+\tgit checkout -b d-chain HEAD &&\n+\ttest_commit d1 &&\n+\ttest_commit B &&\n+\n+\tgit checkout -b e-path C &&\n+\ttest_commit e1 &&\n+\ttest_commit e2 &&\n+\n+\tgit checkout -b f-path C &&\n+\ttest_commit f1 &&\n+\ttest_commit f2 &&\n+\n+\tgit checkout -b branch-A e-path &&\n+\ttest_merge A B &&\n+\n+\tgit checkout -b branch-X f-path &&\n+\ttest_merge X B &&\n+\n+\tgit commit-graph write --reachable\n+'\n+\n+test_expect_success 'merge-base --all excludes ancestor candidate' '\n+\tgit rev-parse B >expected &&\n+\tgit merge-base --all A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'merge-base (single) finds shallowest' '\n+\tgit rev-parse B >expected &&\n+\tgit merge-base A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+# Without commit-graph: generation numbers are INFINITY,\n+# side-exhaustion optimization does not fire.\n+test_expect_success 'merge-base --all without commit-graph' '\n+\trm -f .git/objects/info/commit-graph &&\n+\tgit rev-parse B >expected &&\n+\tgit merge-base --all A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 775c077c87..f5560b0c1c 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -294,6 +294,31 @@ test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'setup mixed finite/INFINITY topology' '\n+\t# Create a commit outside all saved commit-graph files so it always\n+\t# has INFINITY generation, while its parent (ps-X) is in the graph\n+\t# with a finite generation. Use the ps-* orphan topology so we do\n+\t# not pollute the grid-based rev-list tests.\n+\tgit checkout ps-X &&\n+\ttest_env GIT_TEST_COMMIT_GRAPH= test_commit pm-INF\n+'\n+\n+test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n+\t# One tip (pm-INF) is outside the commit-graph with INFINITY\n+\t# generation; the other (ps-B) is in the graph with finite\n+\t# generation. The walk starts in the INFINITY region and crosses\n+\t# into the finite region where side-exhaustion can fire.\n+\tcat >input <<-\\EOF &&\n+\tA:pm-INF\n+\tX:ps-B\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse ps-X\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546027","messageId":"9cbfc67d724d91b9abc3621f03a3c97208c76a70.1781951820.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.git.1781951820.gitgitgadget@gmail.com","subject":"[PATCH/RFC 6/6] Documentation/technical: add paint-down-to-common doc","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-20T10:36:59Z","receivedAt":"2026-06-20T10:37:14Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a technical document describing the paint_down_to_common()\nalgorithm used for merge-base computation.\n\nSigned-off-by: Kristofer Karlsson \n---\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 130 ++++++++++++++++++\n 3 files changed, 132 insertions(+)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 2699f0b24a..f8dea4b395 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -129,6 +129,7 @@ TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\n TECH_DOCS += technical/packfile-uri\n TECH_DOCS += technical/pack-heuristics\n+TECH_DOCS += technical/paint-down-to-common\n TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/platform-support\ndiff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\nindex ec07088c57..9ce11d5e48 100644\n--- a/Documentation/technical/meson.build\n+++ b/Documentation/technical/meson.build\n@@ -18,6 +18,7 @@ articles = [\n 'multi-pack-index.adoc',\n 'packfile-uri.adoc',\n 'pack-heuristics.adoc',\n+ 'paint-down-to-common.adoc',\n 'parallel-checkout.adoc',\n 'partial-clone.adoc',\n 'platform-support.adoc',\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nnew file mode 100644\nindex 0000000000..e677cce84d\n--- /dev/null\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -0,0 +1,130 @@\n+Merge-Base Computation and paint_down_to_common()\n+==================================================\n+\n+The function `paint_down_to_common()` in `commit-reach.c` computes merge\n+bases by walking the commit graph backwards from two sets of tips and\n+finding where their ancestry meets.\n+\n+Use cases\n+---------\n+\n+Computing merge bases is used in two different ways:\n+\n+ 1. *Finding all merge bases* (`merge-base --all`, `merge-tree`,\n+ `merge`, `rebase`). A merge base is a common ancestor that is\n+ not itself an ancestor of another common ancestor.\n+\n+ 2. *Ancestry checks* (`in_merge_bases`, used by `merge-base\n+ --is-ancestor`, `branch -d`, `fetch`). These ask: \"is commit A\n+ an ancestor of commit B?\" If a common ancestor equals one of the\n+ inputs, that input is necessarily the only merge base -- no other\n+ common ancestor can be both as recent and not an ancestor of it.\n+\n+Both use cases share the same algorithm and implementation.\n+\n+Algorithm\n+---------\n+\n+Given a commit `one` and a set of commits `twos[]`, the walk paints\n+commits with two colors:\n+\n+ - PARENT1: reachable from `one`\n+ - PARENT2: reachable from any commit in `twos[]`\n+\n+The walk uses a priority queue ordered by generation number (falling\n+back to commit date when generation numbers are unavailable). Each\n+step dequeues the highest-priority commit (this is when we say a\n+commit is \"visited\") and propagates its paint flags to its parents,\n+enqueuing them if they gained new flags. When a commit receives\n+both PARENT1 and PARENT2, it is a merge-base candidate. A candidate\n+gains the STALE flag so its ancestors propagate staleness -- any\n+deeper common ancestor is necessarily redundant.\n+\n+INFINITY and finite generation regions\n+--------------------------------------\n+\n+The commit-graph stores a generation number for each commit. Commits\n+not in the commit-graph have generation `GENERATION_NUMBER_INFINITY`. The\n+graph is closed under reachability: if a commit is in the graph, all\n+its ancestors are too. This partitions the commit graph into two regions:\n+\n+....\n+ +---------------------------------------+\n+ | INFINITY region |\n+ | generation = INFINITY |\n+ | queue order: heuristic (commit date) |\n+ +---------------------------------------+\n+ |\n+ v\n+ +---------------------------------------+\n+ | Finite region |\n+ | generation = finite |\n+ | queue order: topological |\n+ +---------------------------------------+\n+....\n+\n+When the commit-graph is enabled, the INFINITY region is typically\n+very small -- it only contains commits added since the last\n+commit-graph refresh.\n+\n+All reachable INFINITY-generation commits are visited before any\n+finite-generation commit, because INFINITY is larger than any finite\n+value. Once the walk crosses into the finite region, it stays there.\n+\n+In the finite region, generation ordering guarantees topological\n+traversal: children are always visited before their parents. This\n+means that paint on already-visited commits is final -- no future\n+traversal step can add paint to them.\n+\n+In the INFINITY region, commit-date ordering can violate this: a\n+parent with a later date can be visited before a child with an earlier\n+date. Paint flags are therefore NOT final at visit time, and a\n+commit visited with only one side's paint may later gain the other.\n+\n+Paint flags are only added, never removed. Since each flag can be set\n+at most once per commit, the number of times a commit can be\n+re-enqueued is bounded by the number of flag transitions.\n+\n+Termination\n+-----------\n+\n+Termination happens when we can prove that no extra progress is\n+possible. We are done with the main loop when one of the following\n+conditions holds:\n+\n+ 1. The queue is empty.\n+ 2. The queue only contains STALE entries.\n+ 3. Side-exhaustion: the walk has reached the finite region and one\n+ of the sides is fully exhausted.\n+\n+The loop waits for all pending merge-base candidates to be popped\n+and recorded before any early exit fires, so no separate drain phase\n+is needed after termination.\n+\n+Stale entry condition\n+~~~~~~~~~~~~~~~~~~~~~\n+If all entries are stale we cannot find any new merge bases since\n+that requires at least one enqueued side node meeting the other side.\n+However, we could still invalidate merge bases (if there are more\n+than one). This is unnecessary since `remove_redundant()` will clean\n+that up as a post-process step.\n+\n+Side-exhaustion\n+~~~~~~~~~~~~~~~\n+A commit is *exclusive* to one side if it carries that side's paint\n+but not the other (e.g. PARENT1 without PARENT2).\n+\n+If we have reached the finite region of the graph, no future\n+traversal step can add paint to an already-visited commit. Thus if\n+there are no exclusive PARENT2 commits in the queue, no additional\n+PARENT2 paint can be introduced into the walk. Even if exclusive\n+PARENT1 commits remain, no new merge-base candidates can be\n+discovered. The same holds symmetrically for PARENT1.\n+\n+This invariant is only valid in the finite region of the graph.\n+\n+Related documentation\n+---------------------\n+\n+ - `Documentation/technical/commit-graph.adoc` -- generation numbers\n+ and the reachability closure property.\n-- \ngitgitgadget\n"},{"id":"546198","messageId":"001e8da6-3232-4cfa-ba6b-35d3489e4779@gmail.com","threadId":"65846","inReplyTo":"5492acda0ad05eab67198880a5262e84a3f22ba6.1781951820.git.gitgitgadget@gmail.com","subject":"Re: [PATCH/RFC 1/6] commit-reach: decouple ahead_behind from nonstale_queue","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T18:00:10Z","receivedAt":"2026-06-22T18:00:13Z","isPatch":true,"body":"On 6/20/2026 6:36 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n> \n> Move ahead_behind() off the shared nonstale_queue abstraction to use\n> a plain prio_queue with a local max_nonstale pointer. The nonstale\n> tracking is inlined into insert_no_dup().\n> \n> This prepares for replacing nonstale_queue with a paint_queue struct\n> that tracks per-side commit counts, which ahead_behind() does not\n> need. No behavior change.\n\nThis change is only needed if we are intending to delete the nonstale\nqueue struct, which is currently happening in your patch 2. But we\nare essentially recreating its logic in a more disjointed way here,\nleaving this code in a worse state.\n\nI'd rather see patch 2 create a _new_ data structure instead of\n_replacing_ one that already works for multiple callers. (It does\ndrop to only one caller, but that seems cleaner to me right now.)\n\nThanks,\n-Stolee\n\n"},{"id":"546200","messageId":"f0c9eb6e-60b1-4eb6-86be-3af4d87afe85@gmail.com","threadId":"65846","inReplyTo":"316e4dfe261043730c77142639f86f5c3cabe370.1781951820.git.gitgitgadget@gmail.com","subject":"Re: [PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T18:10:27Z","receivedAt":"2026-06-22T18:10:29Z","isPatch":true,"body":"On 6/20/2026 6:36 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n\n> +\tif (!(old_paint & STALE)) {\n> +\t\tswitch (old_paint & (PARENT1 | PARENT2)) {\n> +\t\tcase 0: break;\n> +\t\tcase PARENT1: queue->p1_count--; break;\n> +\t\tcase PARENT2: queue->p2_count--; break;\n> +\t\tcase PARENT1 | PARENT2: queue->pending_merge_bases--; break;\n> +\t\tdefault: BUG(\"unexpected paint state\");\n> +\t\t}\n> +\t}\n> +\tif (!(new_paint & STALE)) {\n> +\t\tswitch (new_paint & (PARENT1 | PARENT2)) {\n> +\t\tcase 0: break;\n> +\t\tcase PARENT1: queue->p1_count++; break;\n> +\t\tcase PARENT2: queue->p2_count++; break;\n> +\t\tcase PARENT1 | PARENT2: queue->pending_merge_bases++; break;\n> +\t\tdefault: BUG(\"unexpected paint state\");\n> +\t\t}\n> +\t}\n\nWhile correct and compact, I don't believe that these switch\nstatements follow the coding guidelines. We should split the\nlines appropriately so they are more standard, such as:\n\nif (!(new_paint & STALE)) {\n\tswitch (new_paint & (PARENT1 | PARENT2)) {\n\tcase 0:\n\t\tbreak;\n\n\tcase PARENT1:\n\t\tqueue->p1_count++;\n\t\tbreak;\n\n\tcase PARENT2:\n\t\tqueue->p2_count++;\n\t\tbreak;\n\n\tcase PARENT1 | PARENT2:\n\t\tqueue->pending_merge_bases++;\n\t\tbreak;\n\n\tdefault:\n\t\tBUG(\"unexpected paint state\");\n\t}\n}\n\nAlso: technically \"case 0\" should be a BUG() state, right? We\nshouldn't be walking any commit that isn't reachable from at\nleast one side. (case 0 does happen for old_paint, though.)\n\n> }\n> \n> -static void clear_nonstale_queue(struct nonstale_queue *queue)\n> +static void paint_queue_put(struct paint_queue *queue,\n> +\t\t\t struct commit *c, unsigned add_flags)\n> {\n> -\tclear_prio_queue(&queue->pq);\n> -\tqueue->max_nonstale = NULL;\n> -}\n> +\tunsigned old_flags = c->object.flags;\n> +\tc->object.flags |= add_flags;\n \nDiffs like this are part of the reason I'd like to see a _new_\ndata structure instead of replacing the old one. Keeping the\nold one for ahead_behind seems like a good idea to me, but even\nif we don't land on that end state then deleting the old code\n_after_ adding the new code will make the diff more readable.\n\n> -\tstruct nonstale_queue queue = {\n> -\t\t{ compare_commits_by_gen_then_commit_date }\n> +\tstruct paint_queue queue = {\n> +\t\t.pq = { compare_commits_by_gen_then_commit_date }\n> \t};\n\nI didn't notice when reading the struct definition, but looking at\n'pq' here makes me think that we shouldn't be using that abbreviation\nas it could stand for \"prio_queue\" or \"paint_queue\".\n\n> +\twhile ((commit = paint_queue_get(&queue))) {\n...> +\n> +\t\tif (queue.p1_count + queue.p2_count +\n> +\t\t queue.pending_merge_bases == 0)\n> +\t\t\tbreak;\n> \t}\nWhen possible, I like to try to make loops only have one terminating\ncondition. Should we have paint_queue_get() return NULL when it sees\nthis internal state condition?\n\nAlso, I'd rather see it of the form of (!count) instead of using\naddition to make it clear that we care about each value being zero.\n\nFinally, I think we actually want this case to get the benefit:\n\n\tif ((!queue.p1_count || !queue.p2_count) &&\n\t !queue.pending_merge_bases)\n\t \nI do see that you have this condition in patch 3 with the extra\ndetail that the max generation in the queue is finite. I think this\nis more reason to include this in the data structure method and not\nin the loop.\n\nThanks,\n-Stolee\n\n"},{"id":"546201","messageId":"5c43f6ce-4dfe-47dd-b96a-80de57ecf108@gmail.com","threadId":"65846","inReplyTo":"ed12a5cb5b76925cff08d2ab61efeda382b4477a.1781951820.git.gitgitgadget@gmail.com","subject":"Re: [PATCH/RFC 3/6] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T18:12:20Z","receivedAt":"2026-06-22T18:12:22Z","isPatch":true,"body":"On 6/20/2026 6:36 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n> \n> Add an early termination check to paint_down_to_common() using the\n> per-side counters introduced in the previous commit. Once the walk\n> enters the finite-generation region, terminate early when one side's\n> exclusive count drops to zero -- no new merge-base can form without\n> both paint sides meeting.\n> \n> The check also waits for pending_merge_bases to reach zero, ensuring\n> all merge-base candidates have been popped and recorded before\n> exiting.\n> \n> The INFINITY gate ensures correctness: commits without a commit-graph\n> entry have GENERATION_NUMBER_INFINITY and are ordered by commit date,\n> which is not topologically reliable. The optimization only fires\n> once the walk enters the finite-generation region where ordering\n> guarantees hold.\n> \n> On large repositories with commit-graph, this yields 100-1000x\n> speedups for merge-base queries where one side (e.g. a PR branch) is\n> much smaller than the other.\n> \n> Helped-by: Derrick Stolee \n> Helped-by: Elijah Newren \n> Signed-off-by: Kristofer Karlsson \n> ---\n> commit-reach.c | 13 +++++++++++++\n> 1 file changed, 13 insertions(+)\n> \n> diff --git a/commit-reach.c b/commit-reach.c\n> index ba1e896f0f..fcd1ad0167 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -201,6 +201,19 @@ static int paint_down_to_common(struct repository *r,\n> \t\tif (queue.p1_count + queue.p2_count +\n> \t\t queue.pending_merge_bases == 0)\n> \t\t\tbreak;\n> +\n> +\t\t/*\n> +\t\t * Side exhaustion: a new merge-base can only form\n> +\t\t * when both PARENT1-only and PARENT2-only commits\n> +\t\t * remain in the queue. In the finite-generation\n> +\t\t * region the queue is ordered topologically, so\n> +\t\t * no future step can add paint to visited commits\n> +\t\t * and an exhausted side cannot reappear.\n> +\t\t */\n> +\t\tif (generation < GENERATION_NUMBER_INFINITY &&\n> +\t\t queue.pending_merge_bases == 0 &&\n> +\t\t (queue.p1_count == 0 || queue.p2_count == 0))\n> +\t\t\tbreak;\nI mentioned it earlier, but I think this check should be in the\ndequeueing method instead of in the tail of the loop.\n\nBut I think this is the correct ending case.\n\nI like that you broke this out into its own patch to demonstrate\nthat this is the key performance boost. It may be good to have\nsome performance test numbers that demonstrate that patch 2 does\nnot add any substantial overhead (timing should match previous\ncode) and in patch 3 this single condition gets us a huge benefit,\nthough it requires the data tracking of patch 2 to work.\n\nThanks,\n-Stolee\n\n"},{"id":"546202","messageId":"1588b53d-9576-4752-9459-da48276e4b2a@gmail.com","threadId":"65846","inReplyTo":"91372b975fbe102538c05c7d2cdae356539d1bbd.1781951820.git.gitgitgadget@gmail.com","subject":"Re: [PATCH/RFC 4/6] t6600: add test cases for side-exhaustion edge cases","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T18:15:04Z","receivedAt":"2026-06-22T18:15:06Z","isPatch":true,"body":"On 6/20/2026 6:36 AM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Add test cases to t6600-test-reach.sh that exercise edge cases in the\n> side-exhaustion optimization for paint_down_to_common():\n> \n> - in_merge_bases_many:self: commit is both A and one of the X inputs\n> - get_merge_bases_many:duplicate-twos: duplicate entries in X list\n> - get_merge_bases_many:pending-stale: STALE transition on an\n> already-painted commit (ps-* diamond topology)\n> - get_merge_bases_many:infinity-both-sides: both tips outside the\n> commit-graph with non-monotonic dates (pi-* topology)\n\nIt's usually my preference to see these tests show up before the\nnew code arrives, that way we can see that they already work with\nthe old logic and continue to work with the new logic.\n\nIt's minor, but putting them after your code change may be adding\nenforcement of a change of behavior.\n\nOne thing that could be helpful here is to consider tracing a\ncount of \"commits walked\" in the merge-base code, then you could\nhave these tests demonstrate the performance benefit by checking\nfor that number changing.\n\nIn t6600, that tracing number would not be the same across the\nthree different data shapes (full graph, half graph, no graph) and\nthat could be valuable to demonstrate in tests.\n\nThanks,\n-Stolee\n\n"},{"id":"546203","messageId":"f971ace1-5e4c-43e9-a435-56bd76fdc1c9@gmail.com","threadId":"65846","inReplyTo":"faf5bc98ede79965e23bfe1535127d6f52221680.1781951820.git.gitgitgadget@gmail.com","subject":"Re: [PATCH/RFC 5/6] t6099, t6600: add side-exhaustion regression tests","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T18:16:00Z","receivedAt":"2026-06-22T18:16:03Z","isPatch":true,"body":"On 6/20/2026 6:36 AM, Kristofer Karlsson via GitGitGadget wrote:\n> Add t6099 to test the case where multiple merge-base candidates exist\n> and one is an ancestor of another. This exercises the side-exhaustion\n> optimization in paint_down_to_common together with the\n> remove_redundant safety net in get_merge_bases_many_0.\n\nSame as the previous patch: I'd like to see these before the code\nchange. And if we trace a count of commits walked, we'd be able to\nsee the number change in this specific case.\n\nThanks,\n-Stolee\n\n"},{"id":"546204","messageId":"50dd5fb1-6b4e-448c-977c-cdc476f7fe40@gmail.com","threadId":"65846","inReplyTo":"9cbfc67d724d91b9abc3621f03a3c97208c76a70.1781951820.git.gitgitgadget@gmail.com","subject":"Re: [PATCH/RFC 6/6] Documentation/technical: add paint-down-to-common doc","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T18:21:11Z","receivedAt":"2026-06-22T18:21:13Z","isPatch":true,"body":"On 6/20/2026 6:36 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n> \n> Add a technical document describing the paint_down_to_common()\n> algorithm used for merge-base computation.\n\nI like the idea of documenting this so it's easier to understand.\n\nThere is risk of drift from the actual implementation. You may want\nto add a comment to the method in commit-reach.c to indicate that\nany change should be reflected in this document.\n\n> +Termination\n> +-----------\n> +\n> +Termination happens when we can prove that no extra progress is\n> +possible. We are done with the main loop when one of the following\n> +conditions holds:\n> +\n> + 1. The queue is empty.\n> + 2. The queue only contains STALE entries.\n> + 3. Side-exhaustion: the walk has reached the finite region and one\n> + of the sides is fully exhausted.\nIt could be an interesting exercise, but potentially wasteful, to\nadd this document as a Patch 1, but reflecting the old algorithm\nand then to update the document at the same time as you update the\ncode.\n\nThe changes in your patch 2 would impact this doc in terms of the\ndata being tracked by the paint_queue data structure instead of the\nnonstale_queue structure (though those details are not currently\nhandled in the current version). The change to the termination\ncondition would come along with patch 3.\n\nThanks,\n-Stolee\n\n"},{"id":"546205","messageId":"a800f1cf-0ba8-49c4-acb2-40b370e33663@gmail.com","threadId":"65846","inReplyTo":"pull.2149.git.1781951820.gitgitgadget@gmail.com","subject":"Re: [PATCH/RFC 0/6] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T18:22:22Z","receivedAt":"2026-06-22T18:22:24Z","isPatch":true,"body":"On 6/20/2026 6:36 AM, Kristofer Karlsson via GitGitGadget wrote:\n> Hi,\n> \n> This follows up on my RFC [1] with a concrete proposal. I expect the design\n> to still be scrutinized, but that may be easier with actual code to look at.\n> \n> I tried to make this easier to review by splitting into atomic patches. The\n> first two patches are the meatiest parts, though they are pure refactoring.\n> The behavior change is in patch 3 and is in itself quite small. The last\n> patch adds technical documentation to support future development.\nThanks for putting this together carefully.\n\nI gave some feedback on the specific code and the patch organization.\nOverall, I believe that this implementation is functionally correct\nand everything I have to say is about presentation and data gathering.\n\nI look forward to a non-RFC v2.\n\nThanks,\n-Stolee\n"},{"id":"546206","messageId":"CAL71e4MuRXDs_vB7UWq7kXmUVhDHOCv2jKM0NeAj3Cofbt5hcA@mail.gmail.com","threadId":"65846","inReplyTo":"001e8da6-3232-4cfa-ba6b-35d3489e4779@gmail.com","subject":"Re: [PATCH/RFC 1/6] commit-reach: decouple ahead_behind from nonstale_queue","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-22T18:53:09Z","receivedAt":"2026-06-22T18:53:22Z","isPatch":true,"body":"On Mon, 22 Jun 2026 at 20:00, Derrick Stolee wrote:\n>\n> This change is only needed if we are intending to delete the nonstale\n> queue struct, which is currently happening in your patch 2. But we\n> are essentially recreating its logic in a more disjointed way here,\n> leaving this code in a worse state.\n>\n> I'd rather see patch 2 create a _new_ data structure instead of\n> _replacing_ one that already works for multiple callers. (It does\n> drop to only one caller, but that seems cleaner to me right now.)\n\nI can definitely do that and leave ahead_behind unchanged for v2.\nI was thinking that with only a single caller, and ahead_behind\nbeing simpler than paint_down in this respect, it would be\nworthwhile to simplify it, but if so I could instead do that as\na standalone follow up (though it may prove to be not enough\nvalue for the win).\n\nThanks,\nKristofer\n"},{"id":"546207","messageId":"CAL71e4Pcw-UUbHBw_j6PFx2bXmxZ93VLMWG+3Qap=RmCJa_ZgA@mail.gmail.com","threadId":"65846","inReplyTo":"f0c9eb6e-60b1-4eb6-86be-3af4d87afe85@gmail.com","subject":"Re: [PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-22T19:14:57Z","receivedAt":"2026-06-22T19:15:12Z","isPatch":true,"body":"On Mon, 22 Jun 2026 at 20:10, Derrick Stolee wrote:\n>\n> On 6/20/2026 6:36 AM, Kristofer Karlsson via GitGitGadget wrote:\n> > From: Kristofer Karlsson \n>\n> > + if (!(old_paint & STALE)) {\n> > + switch (old_paint & (PARENT1 | PARENT2)) {\n> > + case 0: break;\n> > + case PARENT1: queue->p1_count--; break;\n> > + case PARENT2: queue->p2_count--; break;\n> > + case PARENT1 | PARENT2: queue->pending_merge_bases--; break;\n> > + default: BUG(\"unexpected paint state\");\n> > + }\n> > + }\n> > + if (!(new_paint & STALE)) {\n> > + switch (new_paint & (PARENT1 | PARENT2)) {\n> > + case 0: break;\n> > + case PARENT1: queue->p1_count++; break;\n> > + case PARENT2: queue->p2_count++; break;\n> > + case PARENT1 | PARENT2: queue->pending_merge_bases++; break;\n> > + default: BUG(\"unexpected paint state\");\n> > + }\n> > + }\n>\n> While correct and compact, I don't believe that these switch\n> statements follow the coding guidelines. We should split the\n> lines appropriately so they are more standard, such as:\n>\n> if (!(new_paint & STALE)) {\n> switch (new_paint & (PARENT1 | PARENT2)) {\n> case 0:\n> break;\n>\n> case PARENT1:\n> queue->p1_count++;\n> break;\n>\n> case PARENT2:\n> queue->p2_count++;\n> break;\n>\n> case PARENT1 | PARENT2:\n> queue->pending_merge_bases++;\n> break;\n>\n> default:\n> BUG(\"unexpected paint state\");\n> }\n> }\n\nAgreed, I will change to that style. I did try to look for style guidelines\nbut I missed the .clang-format file (I was only looking through text files).\nApologies, will remember clang-format for next time (and v2)\n\n> Also: technically \"case 0\" should be a BUG() state, right? We\n> shouldn't be walking any commit that isn't reachable from at\n> least one side. (case 0 does happen for old_paint, though.)\n\nNo, this is actually intended - initially I started with skipping\ncase 0 and let it fall through, but that would hide _other_ bugs.\nI use 0 as a marker for \"not in the queue\" so we have this:\nEnqueuing: 0 -> flags\nDequeueing: flags -> 0\nOnly the case with the modified commit being in the queue\nwill have non-zero flags. I tried to document this, but perhaps\nit is not clear enough, I will see if I can rephrase it, or add an\ninline comment around the case itself.\n\n> > -static void clear_nonstale_queue(struct nonstale_queue *queue)\n> > +static void paint_queue_put(struct paint_queue *queue,\n> > + struct commit *c, unsigned add_flags)\n> > {\n> > - clear_prio_queue(&queue->pq);\n> > - queue->max_nonstale = NULL;\n> > -}\n> > + unsigned old_flags = c->object.flags;\n> > + c->object.flags |= add_flags;\n>\n> Diffs like this are part of the reason I'd like to see a _new_\n> data structure instead of replacing the old one. Keeping the\n> old one for ahead_behind seems like a good idea to me, but even\n> if we don't land on that end state then deleting the old code\n> _after_ adding the new code will make the diff more readable.\n\nAgreed, will address that.\n\n> > - struct nonstale_queue queue = {\n> > - { compare_commits_by_gen_then_commit_date }\n> > + struct paint_queue queue = {\n> > + .pq = { compare_commits_by_gen_then_commit_date }\n> > };\n>\n> I didn't notice when reading the struct definition, but looking at\n> 'pq' here makes me think that we shouldn't be using that abbreviation\n> as it could stand for \"prio_queue\" or \"paint_queue\".\n\nGood point, I should pick a longer name for the field. Perhaps simply queue\n(I want to avoid prio_queue since it exactly matches the name of the struct\nwhich could be confusing.)\n\n> > + while ((commit = paint_queue_get(&queue))) {\n> ...> +\n> > + if (queue.p1_count + queue.p2_count +\n> > + queue.pending_merge_bases == 0)\n> > + break;\n> > }\n> When possible, I like to try to make loops only have one terminating\n> condition. Should we have paint_queue_get() return NULL when it sees\n> this internal state condition?\n\nPossibly, but that would couple the paint_queue struct very tightly with\nthe usage. Not a problem in practice since it only has one call site, and\nit's unlikely that we want to add more of them but it may feel more natural\nto let the paint_queue purely have the queue semantics and counters,\nand keep the halt condition within the function itself. I don't feel\nsuper-strongly about this and can change it if needed, I will just need to\nverify that nothing else gets complex as a result, I have not fully thought\nthrough the effects.\n\n> Also, I'd rather see it of the form of (!count) instead of using\n> addition to make it clear that we care about each value being zero.\n\nI did consider that, and most of the code in commit-reach.c at least\nprefers x and !x over x != 0 and x == 0, but my thinking was that\nother code in the repo did use comparison operators specifically\nfor things like counters. Happy to change it to conform better though!\n\n> Finally, I think we actually want this case to get the benefit:\n>\n> if ((!queue.p1_count || !queue.p2_count) &&\n> !queue.pending_merge_bases)\n>\n> I do see that you have this condition in patch 3 with the extra\n> detail that the max generation in the queue is finite. I think this\n> is more reason to include this in the data structure method and not\n> in the loop.\n\nYes, but just to be clear, you don't want to merge together patch 2 and 3\nhere, just grouping the halt conditions closer together\n(within paint_queue_get)? Keeping patch 2 and 3 separate would be nice\nto make it easier to show that introducing this extra counter bookkeeping\ndoes not negatively impact the overall performance too much.\n\nThanks! I appreciate the thorough review of this patch\n(which I feared was the most annoying one to look at).\n\nKristofer\n"},{"id":"546208","messageId":"CAL71e4NJZ9c_=0W4djRFCYPw4z_dkh_ZHEDWBk8cuwXhxT9jgw@mail.gmail.com","threadId":"65846","inReplyTo":"5c43f6ce-4dfe-47dd-b96a-80de57ecf108@gmail.com","subject":"Re: [PATCH/RFC 3/6] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-22T19:19:48Z","receivedAt":"2026-06-22T19:20:01Z","isPatch":true,"body":"On Mon, 22 Jun 2026 at 20:12, Derrick Stolee wrote:\n> > + if (generation < GENERATION_NUMBER_INFINITY &&\n> > + queue.pending_merge_bases == 0 &&\n> > + (queue.p1_count == 0 || queue.p2_count == 0))\n> > + break;\n> I mentioned it earlier, but I think this check should be in the\n> dequeueing method instead of in the tail of the loop.\n\nYes, I will try to fold this one into the paint_queue_get as well.\n\n> I like that you broke this out into its own patch to demonstrate\n> that this is the key performance boost. It may be good to have\n> some performance test numbers that demonstrate that patch 2 does\n> not add any substantial overhead (timing should match previous\n> code) and in patch 3 this single condition gets us a huge benefit,\n> though it requires the data tracking of patch 2 to work.\n\nGood point, I will try to run enough local tests to ensure that patch 2\ndoes not add too much overhead to slow things down.\nI think I may need to create some type of (temporary, internal)\ntest runner that runs the same walk multiple times to reduce\nthe noise from parsing commits. I am not sure if I should also\ncommit such a performance test or simply include a brief summary\nin the commit message\n\nThanks,\nKristofer\n"},{"id":"546209","messageId":"CAL71e4M0T4fFG4JuYTp_ZPHzNcHXf342Xkh0n0dt4LVKsuSu2Q@mail.gmail.com","threadId":"65846","inReplyTo":"1588b53d-9576-4752-9459-da48276e4b2a@gmail.com","subject":"Re: [PATCH/RFC 4/6] t6600: add test cases for side-exhaustion edge cases","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-22T19:25:15Z","receivedAt":"2026-06-22T19:25:28Z","isPatch":true,"body":"On Mon, 22 Jun 2026 at 20:15, Derrick Stolee wrote:\n> It's usually my preference to see these tests show up before the\n> new code arrives, that way we can see that they already work with\n> the old logic and continue to work with the new logic.\n>\n> It's minor, but putting them after your code change may be adding\n> enforcement of a change of behavior.\n\nAgreed, I actually also prefer that in practice so I am not\nsure why I ordered them this way - perhaps some attempt at\nmaking it easier to review (show the idea and change before\nthe verification). I will reorder to put all new tests as the first commit\n(or second, if I will also introduce a status-quo technical first).\n\n>\n> One thing that could be helpful here is to consider tracing a\n> count of \"commits walked\" in the merge-base code, then you could\n> have these tests demonstrate the performance benefit by checking\n> for that number changing.\n\nGood idea, I actually had some of that locally when developing it,\nbut I removed the ugly traces before submitting this. I will try to\nre-introduce that in a nice way. It would be neat to let tests\ninspect that side effect, though in the worst case that could make\nit fragile. At the very least it's good for human debugging though.\n\n> In t6600, that tracing number would not be the same across the\n> three different data shapes (full graph, half graph, no graph) and\n> that could be valuable to demonstrate in tests.\n\nAgreed, the number of commits visited would be more interesting\nthan the relative performance numbers since it's an algorithmic\nchange rather than a micro-optimization.\n\nThanks,\nKristofer\n"},{"id":"546210","messageId":"CAL71e4PhazEZjaO9nY1SsjbYH+RPAOpJiUo4xSCnBCA6zi6ySQ@mail.gmail.com","threadId":"65846","inReplyTo":"50dd5fb1-6b4e-448c-977c-cdc476f7fe40@gmail.com","subject":"Re: [PATCH/RFC 6/6] Documentation/technical: add paint-down-to-common doc","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-22T19:30:16Z","receivedAt":"2026-06-22T19:30:29Z","isPatch":true,"body":"On Mon, 22 Jun 2026 at 20:21, Derrick Stolee wrote:\n>\n> I like the idea of documenting this so it's easier to understand.\n\nYes I was myself thinking that I can prove it to myself now that it works,\nand anyone else could also prove it to themselves, but having it\nexplicit here is even better. I found the other documents\n(i.e. commit-graph) to be a good source of inspiration here.\n\n> There is risk of drift from the actual implementation. You may want\n> to add a comment to the method in commit-reach.c to indicate that\n> any change should be reflected in this document.\n\nGood idea, will add that.\n\n> > +Termination\n> > +-----------\n> > +\n> > +Termination happens when we can prove that no extra progress is\n> > +possible. We are done with the main loop when one of the following\n> > +conditions holds:\n> > +\n> > + 1. The queue is empty.\n> > + 2. The queue only contains STALE entries.\n> > + 3. Side-exhaustion: the walk has reached the finite region and one\n> > + of the sides is fully exhausted.\n> It could be an interesting exercise, but potentially wasteful, to\n> add this document as a Patch 1, but reflecting the old algorithm\n> and then to update the document at the same time as you update the\n> code.\n\nI did consider that initially but I was worried it would be considered\nnoisy. I am quite happy to rework it in a way that first\nexplains the status quo. That would make the document diff\nmore interesting. Agreed that should become the first patch,\nand the patch that changes the algorithm should include\nthe documentation change.\n\n> The changes in your patch 2 would impact this doc in terms of the\n> data being tracked by the paint_queue data structure instead of the\n> nonstale_queue structure (though those details are not currently\n> handled in the current version). The change to the termination\n> condition would come along with patch 3.\n\nAgreed, I would need to rephrase from tracking non-stale\nto tracking counts of p1 and p2 (and pending merge bases) commits,\nbut I think that would be a small tweak and well worth doing.\n\nThanks,\nKristofer\n"},{"id":"546212","messageId":"8d07f5a9-82fa-4aed-b407-363e659f6851@gmail.com","threadId":"65846","inReplyTo":"CAL71e4Pcw-UUbHBw_j6PFx2bXmxZ93VLMWG+3Qap=RmCJa_ZgA@mail.gmail.com","subject":"Re: [PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T20:23:25Z","receivedAt":"2026-06-22T20:23:27Z","isPatch":true,"body":"On 6/22/2026 3:14 PM, Kristofer Karlsson wrote:\n> On Mon, 22 Jun 2026 at 20:10, Derrick Stolee wrote:\n>>\n\n>> Also: technically \"case 0\" should be a BUG() state, right? We\n>> shouldn't be walking any commit that isn't reachable from at\n>> least one side. (case 0 does happen for old_paint, though.)\n> \n> No, this is actually intended - initially I started with skipping\n> case 0 and let it fall through, but that would hide _other_ bugs.\n> I use 0 as a marker for \"not in the queue\" so we have this:\n> Enqueuing: 0 -> flags\n> Dequeueing: flags -> 0\n> Only the case with the modified commit being in the queue\n> will have non-zero flags. I tried to document this, but perhaps\n> it is not clear enough, I will see if I can rephrase it, or add an\n> inline comment around the case itself.\n\nI bet this would be obvious if I tried to change the code and\nrun the tests. thanks for the explanation.\n\n>>> + while ((commit = paint_queue_get(&queue))) {\n>> ...> +\n>>> + if (queue.p1_count + queue.p2_count +\n>>> + queue.pending_merge_bases == 0)\n>>> + break;\n>>> }\n>> When possible, I like to try to make loops only have one terminating\n>> condition. Should we have paint_queue_get() return NULL when it sees\n>> this internal state condition?\n> \n> Possibly, but that would couple the paint_queue struct very tightly with\n> the usage. Not a problem in practice since it only has one call site, and\n> it's unlikely that we want to add more of them but it may feel more natural\n> to let the paint_queue purely have the queue semantics and counters,\n> and keep the halt condition within the function itself. I don't feel\n> super-strongly about this and can change it if needed, I will just need to\n> verify that nothing else gets complex as a result, I have not fully thought\n> through the effects.\n\nHm. Interesting. The coupling is perhaps expected, because the data\nstructure tracks counts that don't otherwise need to be tracked.\nMaybe the terminating condition method could be descriptively named\nto say why it would be completing.\n\n>> Also, I'd rather see it of the form of (!count) instead of using\n>> addition to make it clear that we care about each value being zero.\n> \n> I did consider that, and most of the code in commit-reach.c at least\n> prefers x and !x over x != 0 and x == 0, but my thinking was that\n> other code in the repo did use comparison operators specifically\n> for things like counters. Happy to change it to conform better though!\nI just worry about the idea that a negative number (or an addition\noverflow) would create conditions for termination that we did not\nintend. That's why using the nonzero status as true/false combined\nwith ands and ors is better.\n\n>> Finally, I think we actually want this case to get the benefit:\n>>\n>> if ((!queue.p1_count || !queue.p2_count) &&\n>> !queue.pending_merge_bases)\n>>\n>> I do see that you have this condition in patch 3 with the extra\n>> detail that the max generation in the queue is finite. I think this\n>> is more reason to include this in the data structure method and not\n>> in the loop.\n> \n> Yes, but just to be clear, you don't want to merge together patch 2 and 3\n> here, just grouping the halt conditions closer together\n> (within paint_queue_get)? Keeping patch 2 and 3 separate would be nice\n> to make it easier to show that introducing this extra counter bookkeeping\n> does not negatively impact the overall performance too much.\nNo, I don't want you to squash them. I was perhaps unclear as I was\ndiscovering the structure as we went. The thing I was missing above\nwas the \"finite generation number\" condition, which you make very\nclear in patch 3.\n\nThanks,\n-Stolee\n\n\n"},{"id":"546213","messageId":"4f9cae3c-5cef-420b-954b-d1981d9d5a67@gmail.com","threadId":"65846","inReplyTo":"CAL71e4NJZ9c_=0W4djRFCYPw4z_dkh_ZHEDWBk8cuwXhxT9jgw@mail.gmail.com","subject":"Re: [PATCH/RFC 3/6] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T20:26:00Z","receivedAt":"2026-06-22T20:26:02Z","isPatch":true,"body":"On 6/22/2026 3:19 PM, Kristofer Karlsson wrote:\n\n> I think I may need to create some type of (temporary, internal)\n> test runner that runs the same walk multiple times to reduce\n> the noise from parsing commits.\nI've used hyperfine [1] when doing specific performance tests\nin the past. You can build Git before and after and have hyperfine\nrun the two modes and compare them:\n\n\thyperfine --warmup=3 \\\n\t\t-n 'old' \"~/git-old/bin-wrappers/git -C $repo merge-base $A $B\" \\\n\t\t-n 'new' \"~/git-new/bin-wrappers/git -C $repo merge-base $A $B\"\n\n[1] https://github.com/sharkdp/hyperfine\n\nGood luck!\n-Stolee\n\n"},{"id":"546214","messageId":"80a0426c-7146-4212-a8cd-d884f4424b2c@gmail.com","threadId":"65846","inReplyTo":"CAL71e4M0T4fFG4JuYTp_ZPHzNcHXf342Xkh0n0dt4LVKsuSu2Q@mail.gmail.com","subject":"Re: [PATCH/RFC 4/6] t6600: add test cases for side-exhaustion edge cases","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-22T20:28:22Z","receivedAt":"2026-06-22T20:28:25Z","isPatch":true,"body":"On 6/22/2026 3:25 PM, Kristofer Karlsson wrote:\n> On Mon, 22 Jun 2026 at 20:15, Derrick Stolee wrote:\n>> It's usually my preference to see these tests show up before the\n>> new code arrives, that way we can see that they already work with\n>> the old logic and continue to work with the new logic.\n>>\n>> It's minor, but putting them after your code change may be adding\n>> enforcement of a change of behavior.\n> \n> Agreed, I actually also prefer that in practice so I am not\n> sure why I ordered them this way - perhaps some attempt at\n> making it easier to review (show the idea and change before\n> the verification). I will reorder to put all new tests as the first commit\n> (or second, if I will also introduce a status-quo technical first).\n> \n>>\n>> One thing that could be helpful here is to consider tracing a\n>> count of \"commits walked\" in the merge-base code, then you could\n>> have these tests demonstrate the performance benefit by checking\n>> for that number changing.\n> \n> Good idea, I actually had some of that locally when developing it,\n> but I removed the ugly traces before submitting this. I will try to\n> re-introduce that in a nice way. It would be neat to let tests\n> inspect that side effect, though in the worst case that could make\n> it fragile. At the very least it's good for human debugging though.\n\nAnd to be clear, I'm suggesting using trace2_data_intmax() calls\nto get structured data that can be parsed in the GIT_TRACE2_EVENT\nlogs during tests. It could also be picked up by teletry tools that\nlisten to trace2 output, if desired.\n\nIt will show up differently in GIT_TRACE2_PERF, but that's a nice\nhuman-readable way to debug things.\n\n>> In t6600, that tracing number would not be the same across the\n>> three different data shapes (full graph, half graph, no graph) and\n>> that could be valuable to demonstrate in tests.\n> \n> Agreed, the number of commits visited would be more interesting\n> than the relative performance numbers since it's an algorithmic\n> change rather than a micro-optimization.\nThey are both interesting, but only the commit count can be\nguaranteed rigorously in the test suite. It's possible that a\ngreat improvement to such a trace doesn't result in great end-to-\nend time improvement, but I believe that it is true in this case.\n\nThanks,\n-Stolee\n"},{"id":"546217","messageId":"CAL71e4O7hKM=_M4K9hJE0MH9PdHUHxo7hyAbUSLbyk2wpiGxmw@mail.gmail.com","threadId":"65846","inReplyTo":"4f9cae3c-5cef-420b-954b-d1981d9d5a67@gmail.com","subject":"Re: [PATCH/RFC 3/6] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-22T21:03:03Z","receivedAt":"2026-06-22T21:03:16Z","isPatch":true,"body":"On Mon, 22 Jun 2026 at 22:26, Derrick Stolee wrote:\n>\n> I've used hyperfine [1] when doing specific performance tests\n> in the past. You can build Git before and after and have hyperfine\n> run the two modes and compare them:\n>\n> hyperfine --warmup=3 \\\n> -n 'old' \"~/git-old/bin-wrappers/git -C $repo merge-base $A $B\" \\\n> -n 'new' \"~/git-new/bin-wrappers/git -C $repo merge-base $A $B\"\n>\n> [1] https://github.com/sharkdp/hyperfine\n\nI can definitely use that, but I was thinking that the overhead\nof operations such as repo_parse_commit would be high relative\nto the overhead of the new paint_queue struct such that it would\nbe hard to properly measure and that it would be easier if I could\nspread out that cost across multiple internal runs (which requires\na custom binary of some sort), but perhaps it's enough to just\nshow that there's no measurable regression here and then\nhyperfine is indeed the right fit. I'll start with that and see if I need\nto do anything more complex.\n\nThanks,\nKristofer\n"},{"id":"546226","messageId":"CAL71e4NFHz_zVCWPvmTO8UPNyaKkDFqNQdd3CJykoiGmEhfUTA@mail.gmail.com","threadId":"65846","inReplyTo":"8d07f5a9-82fa-4aed-b407-363e659f6851@gmail.com","subject":"Re: [PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-23T10:13:10Z","receivedAt":"2026-06-23T10:13:22Z","isPatch":true,"body":"On Mon, 22 Jun 2026 at 22:23, Derrick Stolee wrote:\n>\n> On 6/22/2026 3:14 PM, Kristofer Karlsson wrote:\n> >\n> > On Mon, 22 Jun 2026 at 20:10, Derrick Stolee wrote:\n> >>\n> >> When possible, I like to try to make loops only have one terminating\n> >> condition. Should we have paint_queue_get() return NULL when it sees\n> >> this internal state condition?\n> >\n> > Possibly, but that would couple the paint_queue struct very tightly with\n> > the usage. Not a problem in practice since it only has one call site, and\n> > it's unlikely that we want to add more of them but it may feel more natural\n> > to let the paint_queue purely have the queue semantics and counters,\n> > and keep the halt condition within the function itself. I don't feel\n> > super-strongly about this and can change it if needed, I will just need to\n> > verify that nothing else gets complex as a result, I have not fully thought\n> > through the effects.\n>\n> Hm. Interesting. The coupling is perhaps expected, because the data\n> structure tracks counts that don't otherwise need to be tracked.\n> Maybe the terminating condition method could be descriptively named\n> to say why it would be completing.\n>\n\nI have been working on v2 locally and most of the changes landed\nnicely and were clear improvements but there's one point I would\nwant to discuss a bit more.\n\nFor the termination conditions, I moved them into paint_queue_get()\nas you suggested. The all-zero check was straightforward since it\nonly depends on the counters but the side-exhaustion check also\nneeds to know whether we have entered the finite-generation region,\nso I pass last_gen (already a local in paint_down_to_common) as a\nparameter:\n\n static struct commit *paint_queue_get(struct paint_state *state,\n timestamp_t last_gen)\n\nInside, the two conditions merge nicely under a shared guard:\n\n if (!state->pending_merge_bases) {\n if (!state->p1_count && !state->p2_count)\n return NULL;\n if (last_gen < GENERATION_NUMBER_INFINITY &&\n (!state->p1_count || !state->p2_count))\n return NULL;\n }\n\nBoth conditions require pending_merge_bases == 0, so the nesting\nfelt natural. The first is \"nothing non-stale left\" (works in any\nregion). The second is \"one side exhausted\" (only in the finite\nregion where topological ordering holds).\n\nI think passing in last_gen into paint_queue_get() feels _slightly_\nawkward but not too bad in practice. However, we also have my\nolder (first) patch with the fast-exit if the caller only needs one\nmerge base -- that has a separate break that also could be folded\ninto paint_queue_get(). The messy part here is that we would need\nto also pass the mb_flags parameter to paint_queue_get().\n\nPerhaps we should just let this remain as-is for now and follow up\nwith _removing_ that optimization. I think the value of having it\nis much diminished (but not fully gone) by the side-exhaust approach.\n\nAdditionally there's a correctness argument to be made -- perhaps\nall callers _should_ care about multiple merge bases existing, and\ninstead bail out if it finds more than one. The only use case\nwhere this matters today is \"git merge-base A B\" without --all.\n\nRight now I am leaning towards simply passing in last_gen and\ncontaining all of the halt conditions there\n(except the old !FIND_ALL).\n\nThe nicest alternative I can think of is to let this part only\nbreak when the queue is empty:\n\n while ((commit = paint_queue_get(&state)))\n\nand then adding a logical halt-section at the end of the while-loop\n(where all the useful variables we need are already available), and\nwe could logically think of that as an optimization section, never\nstrictly needed for correctness.\n\n> I just worry about the idea that a negative number (or an addition\n> overflow) would create conditions for termination that we did not\n> intend. That's why using the nonzero status as true/false combined\n> with ands and ors is better.\n\nGood point, I have addressed that locally too.\n\nThanks,\nKristofer\n"},{"id":"546231","messageId":"82c160f7-94c6-4f38-ad16-1f57abc29da6@gmail.com","threadId":"65846","inReplyTo":"CAL71e4O7hKM=_M4K9hJE0MH9PdHUHxo7hyAbUSLbyk2wpiGxmw@mail.gmail.com","subject":"Re: [PATCH/RFC 3/6] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-23T13:40:49Z","receivedAt":"2026-06-23T13:40:54Z","isPatch":true,"body":"On 6/22/2026 5:03 PM, Kristofer Karlsson wrote:\n> On Mon, 22 Jun 2026 at 22:26, Derrick Stolee wrote:\n>>\n>> I've used hyperfine [1] when doing specific performance tests\n>> in the past. You can build Git before and after and have hyperfine\n>> run the two modes and compare them:\n>>\n>> hyperfine --warmup=3 \\\n>> -n 'old' \"~/git-old/bin-wrappers/git -C $repo merge-base $A $B\" \\\n>> -n 'new' \"~/git-new/bin-wrappers/git -C $repo merge-base $A $B\"\n>>\n>> [1] https://github.com/sharkdp/hyperfine\n> \n> I can definitely use that, but I was thinking that the overhead\n> of operations such as repo_parse_commit would be high relative\n> to the overhead of the new paint_queue struct such that it would\n> be hard to properly measure and that it would be easier if I could\n> spread out that cost across multiple internal runs (which requires\n> a custom binary of some sort), but perhaps it's enough to just\n> show that there's no measurable regression here and then\n> hyperfine is indeed the right fit. I'll start with that and see if I need\n> to do anything more complex.\nUnit-level performance is nice, but doesn't tell the whole story.\n\nWe typically focus on end-to-end performance numbers when possible.\n\nAnother way to do it would be to use trace2_region_enter() and\ntrace2_region_leave() markers and then pull the timing data out of\nthe trace2 event logs. It's more complicated and usually only\nneeded if we are struggling to reproduce the performance impact due\nto external factors.\n\nThanks,\n-Stolee\n\n"},{"id":"546234","messageId":"509fa950-fb9b-468d-b917-6c0eb7823d64@gmail.com","threadId":"65846","inReplyTo":"CAL71e4NFHz_zVCWPvmTO8UPNyaKkDFqNQdd3CJykoiGmEhfUTA@mail.gmail.com","subject":"Re: [PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-23T13:50:13Z","receivedAt":"2026-06-23T13:50:18Z","isPatch":true,"body":"On 6/23/2026 6:13 AM, Kristofer Karlsson wrote:\n> On Mon, 22 Jun 2026 at 22:23, Derrick Stolee wrote:\n>>\n>> On 6/22/2026 3:14 PM, Kristofer Karlsson wrote:\n>>>\n>>> On Mon, 22 Jun 2026 at 20:10, Derrick Stolee wrote:\n>>>>\n>>>> When possible, I like to try to make loops only have one terminating\n>>>> condition. Should we have paint_queue_get() return NULL when it sees\n>>>> this internal state condition?\n>>>\n>>> Possibly, but that would couple the paint_queue struct very tightly with\n>>> the usage. Not a problem in practice since it only has one call site, and\n>>> it's unlikely that we want to add more of them but it may feel more natural\n>>> to let the paint_queue purely have the queue semantics and counters,\n>>> and keep the halt condition within the function itself. I don't feel\n>>> super-strongly about this and can change it if needed, I will just need to\n>>> verify that nothing else gets complex as a result, I have not fully thought\n>>> through the effects.\n>>\n>> Hm. Interesting. The coupling is perhaps expected, because the data\n>> structure tracks counts that don't otherwise need to be tracked.\n>> Maybe the terminating condition method could be descriptively named\n>> to say why it would be completing.\n>>\n> \n> I have been working on v2 locally and most of the changes landed\n> nicely and were clear improvements but there's one point I would\n> want to discuss a bit more.\n> \n> For the termination conditions, I moved them into paint_queue_get()\n> as you suggested. The all-zero check was straightforward since it\n> only depends on the counters but the side-exhaustion check also\n> needs to know whether we have entered the finite-generation region,\n> so I pass last_gen (already a local in paint_down_to_common) as a\n> parameter:\n> \n> static struct commit *paint_queue_get(struct paint_state *state,\n> timestamp_t last_gen)\n> \n> Inside, the two conditions merge nicely under a shared guard:\n> \n> if (!state->pending_merge_bases) {\n> if (!state->p1_count && !state->p2_count)\n> return NULL;\n> if (last_gen < GENERATION_NUMBER_INFINITY &&\n> (!state->p1_count || !state->p2_count))\n> return NULL;\n> }\n\nThis looks good to me. I'm not even bothered by the last_gen\nparameter. You do make a good point about it being a potentially\nleaky abstraction.\n\n> Both conditions require pending_merge_bases == 0, so the nesting\n> felt natural. The first is \"nothing non-stale left\" (works in any\n> region). The second is \"one side exhausted\" (only in the finite\n> region where topological ordering holds).\n> \n> I think passing in last_gen into paint_queue_get() feels _slightly_\n> awkward but not too bad in practice. However, we also have my\n> older (first) patch with the fast-exit if the caller only needs one\n> merge base -- that has a separate break that also could be folded\n> into paint_queue_get(). The messy part here is that we would need\n> to also pass the mb_flags parameter to paint_queue_get().\n\nHow much of this data that you are passing into the method could be\nstate in the paint_queue struct? Could we have the paint_queue manage\nall of the state necessary to make decisions around the walk\ntermination?\n\nOr, could we do a peek into the queue to see the \"top\" commit, and\ncheck if it is a finite commit or not? I know that 'last_gen' is\nsupposed to be the commit walked in the previous cycle, but it seems\nthat we only care about \"the remaining commits are finite\" as our\ncondition. \n> Perhaps we should just let this remain as-is for now and follow up\n> with _removing_ that optimization. I think the value of having it\n> is much diminished (but not fully gone) by the side-exhaust approach.\n> \n> Additionally there's a correctness argument to be made -- perhaps\n> all callers _should_ care about multiple merge bases existing, and\n> instead bail out if it finds more than one. The only use case\n> where this matters today is \"git merge-base A B\" without --all.\n\n> Right now I am leaning towards simply passing in last_gen and\n> containing all of the halt conditions there\n> (except the old !FIND_ALL).\n\nThis is a good start, but hopefully storing the data in the\nstruct would be a good way to handle that.\n\nThanks,\n-Stolee\n\n"},{"id":"546235","messageId":"CAL71e4PzjdNCaVRtXg7wh9s6DxBeA4ock1aTzq8VPxKCmE-obA@mail.gmail.com","threadId":"65846","inReplyTo":"509fa950-fb9b-468d-b917-6c0eb7823d64@gmail.com","subject":"Re: [PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-23T14:09:28Z","receivedAt":"2026-06-23T14:09:41Z","isPatch":true,"body":"On Tue, 23 Jun 2026 at 15:50, Derrick Stolee wrote:\n>\n> > For the termination conditions, I moved them into paint_queue_get()\n> > as you suggested. The all-zero check was straightforward since it\n> > only depends on the counters but the side-exhaustion check also\n> > needs to know whether we have entered the finite-generation region,\n> > so I pass last_gen (already a local in paint_down_to_common) as a\n> > parameter:\n> >\n> > static struct commit *paint_queue_get(struct paint_state *state,\n> > timestamp_t last_gen)\n> >\n> > Inside, the two conditions merge nicely under a shared guard:\n> >\n> > if (!state->pending_merge_bases) {\n> > if (!state->p1_count && !state->p2_count)\n> > return NULL;\n> > if (last_gen < GENERATION_NUMBER_INFINITY &&\n> > (!state->p1_count || !state->p2_count))\n> > return NULL;\n> > }\n>\n> This looks good to me. I'm not even bothered by the last_gen\n> parameter. You do make a good point about it being a potentially\n> leaky abstraction.\n\nAgreed, I am not also bothered by it.\n\n> > Both conditions require pending_merge_bases == 0, so the nesting\n> > felt natural. The first is \"nothing non-stale left\" (works in any\n> > region). The second is \"one side exhausted\" (only in the finite\n> > region where topological ordering holds).\n> >\n> > I think passing in last_gen into paint_queue_get() feels _slightly_\n> > awkward but not too bad in practice. However, we also have my\n> > older (first) patch with the fast-exit if the caller only needs one\n> > merge base -- that has a separate break that also could be folded\n> > into paint_queue_get(). The messy part here is that we would need\n> > to also pass the mb_flags parameter to paint_queue_get().\n>\n> How much of this data that you are passing into the method could be\n> state in the paint_queue struct? Could we have the paint_queue manage\n> all of the state necessary to make decisions around the walk\n> termination?\n\nGood idea, I think adding last_gen to the struct is doable and makes it cleaner.\nIf needed we could also add the mb_flags there (but would be a followup patch)\nMinor note: I renamed the struct to paint_state so that I could rename\nthe prio_queue to queue and not have \"queue.queue\" which felt\nconfusing in the code.\n\n> Or, could we do a peek into the queue to see the \"top\" commit, and\n> check if it is a finite commit or not? I know that 'last_gen' is\n> supposed to be the commit walked in the previous cycle, but it seems\n> that we only care about \"the remaining commits are finite\" as our\n> condition.\n\nYes, peeking into the queue would work too, but it would feel awkward,\n\n commit = prio_queue_peek();\n if (halt conditions) return NULL;\n prio_queue_get();\n\nAnd if we get first, the condition is not valid - that said, it would be doable\nto instead put the halt conditions _between_ popping the commit and\nupdating the counters. I am not sure how ugly or confusing it would be,\nbut I could add a comment to explain why that sequencing is important.\n(Popping the commit and updating the counters may lead to temporary\n0 counts, but then when we enqueue parents of the commits they\nmove away from the 0 anyway). It would become something like:\n\n// dry-/pseudo-coded\n commit *paint_queue_pop() {\n commit = prio_queue_pop();\n if (!commit) return NULL;\n if (halt_condition(state, commit.generation)) return NULL;\n // important: don't decrement counters before checking the halt condition\n paint_count_update(state, commit->object.flags, -1);\n return commit;\n }\n\n> > Right now I am leaning towards simply passing in last_gen and\n> > containing all of the halt conditions there\n> > (except the old !FIND_ALL).\n>\n> This is a good start, but hopefully storing the data in the\n> struct would be a good way to handle that.\n\nSounds good, I will massage the code a bit, store the relevant pieces\nin the struct\nand see how clean I can make it.\n\nThanks,\nKristofer\n"},{"id":"546236","messageId":"ec241a02-546c-4b5f-8ef7-06b4355d8fec@gmail.com","threadId":"65846","inReplyTo":"CAL71e4PzjdNCaVRtXg7wh9s6DxBeA4ock1aTzq8VPxKCmE-obA@mail.gmail.com","subject":"Re: [PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-23T14:17:40Z","receivedAt":"2026-06-23T14:17:42Z","isPatch":true,"body":"On 6/23/2026 10:09 AM, Kristofer Karlsson wrote:\n> On Tue, 23 Jun 2026 at 15:50, Derrick Stolee wrote:\n\n>> How much of this data that you are passing into the method could be\n>> state in the paint_queue struct? Could we have the paint_queue manage\n>> all of the state necessary to make decisions around the walk\n>> termination?\n> \n> Good idea, I think adding last_gen to the struct is doable and makes it cleaner.\n> If needed we could also add the mb_flags there (but would be a followup patch)\n> Minor note: I renamed the struct to paint_state so that I could rename\n> the prio_queue to queue and not have \"queue.queue\" which felt\n> confusing in the code.\n> \n>> Or, could we do a peek into the queue to see the \"top\" commit, and\n>> check if it is a finite commit or not? I know that 'last_gen' is\n>> supposed to be the commit walked in the previous cycle, but it seems\n>> that we only care about \"the remaining commits are finite\" as our\n>> condition.\n> \n> Yes, peeking into the queue would work too, but it would feel awkward,\n> \n> commit = prio_queue_peek();\n> if (halt conditions) return NULL;\n> prio_queue_get();\n\nGood instinct to notice that peeking and getting from the same\nmethod is awkward.\n> And if we get first, the condition is not valid - that said, it would be doable\n> to instead put the halt conditions _between_ popping the commit and\n> updating the counters. I am not sure how ugly or confusing it would be,\n> but I could add a comment to explain why that sequencing is important.\n> (Popping the commit and updating the counters may lead to temporary\n> 0 counts, but then when we enqueue parents of the commits they\n> move away from the 0 anyway). It would become something like:\n> \n> // dry-/pseudo-coded\n> commit *paint_queue_pop() {\n> commit = prio_queue_pop();\n> if (!commit) return NULL;\n> if (halt_condition(state, commit.generation)) return NULL;\n> // important: don't decrement counters before checking the halt condition\n> paint_count_update(state, commit->object.flags, -1);\n> return commit;\n> }\nI think this would be an appropriate way to handle this. If we\npop and return NULL then it's ok that we removed data from the\nqueue because it shouldn't be reused.\n\nThanks,\n-Stolee\n"},{"id":"546289","messageId":"CAL71e4Mx8=JFt+UcqiLN+Wb_UvGvF=SP81Nb3z_VZaEVahBfBg@mail.gmail.com","threadId":"65846","inReplyTo":"ec241a02-546c-4b5f-8ef7-06b4355d8fec@gmail.com","subject":"Re: [PATCH/RFC 2/6] commit-reach: introduce struct paint_queue with per-side counters","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-24T11:25:30Z","receivedAt":"2026-06-24T11:25:43Z","isPatch":true,"body":"On Tue, 23 Jun 2026 at 16:17, Derrick Stolee wrote:\n>\n> I think this would be an appropriate way to handle this. If we\n> pop and return NULL then it's ok that we removed data from the\n> queue because it shouldn't be reused.\n\nI have prepared v2 on GGG which I believe addresses all of the\nfeedback. The halt conditions now live inside paint_queue_get()\nas you suggested.\n\nI am not 100% happy with the halt-condition placement yet --\nthe existing loop in master already has several exit paths\n(while condition, min_generation break, FIND_ALL break) and I\nthink there is an opportunity to consolidate them. But that is\na separate discussion and I do not want to derail this series.\nI can propose some alternatives in a follow-up after this\nlands.\n\nThanks,\nKristofer\n"},{"id":"546300","messageId":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.git.1781951820.gitgitgadget@gmail.com","subject":"[PATCH v2 0/7] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-24T12:14:06Z","receivedAt":"2026-06-24T12:14:17Z","isPatch":true,"body":"commit-reach: terminate merge-base walk when one side is exhausted\n\nOptimize paint_down_to_common() for merge-base queries that hit large\none-sided histories.\n\nWhen the walk from one side reaches a commit with a very low generation\nnumber that the other side never paints, the walk is forced to drain most of\nthe graph. A common trigger is a repository import that grafts a separate\nhistory with its own root, but any merge that introduces a low-generation\ncommit never painted by the other side has the same effect.\n\nA new merge-base candidate can only be discovered when exclusive PARENT1 and\nPARENT2 paint meet. This series teaches paint_down_to_common() to stop as\nsoon as one side has no exclusive commits left in the queue; once one side\nis exhausted, no further candidates can appear.\n\n origin/HEAD o o PR HEAD\n | |\n (import) o :\n / \\ /\n | o merge-base\n | |\n : : (~2.5M commits)\n | |\n import root main root\n\n\nIn the RFC thread [1], Derrick Stolee provided a criss-cross counterexample\nthat sharpened the halt condition, and Elijah Newren independently\ndiscovered the same optimization and shared an implementation in PR #2150\n[2]. Patches 2-3 incorporate test cases from Elijah's branch.\n\nThis series implements the optimization only after the walk enters the\nfinite-generation region, where generation ordering guarantees that paint on\nvisited commits is final.\n\nPatch layout:\n\n1/7 Documentation/technical: add paint-down-to-common doc 2/7 t6600: add\ntest cases for side-exhaustion edge cases 3/7 t6099, t6600: add\nside-exhaustion regression tests 4/7 commit-reach: add trace2\ninstrumentation to paint_down_to_common() 5/7 commit-reach: introduce struct\npaint_state with per-side counters 6/7 commit-reach: remove unused\nnonstale_queue dedup wrappers 7/7 commit-reach: terminate merge-base walk\nwhen one paint side is exhausted\n\nBenchmarks\n\nStep counts are deterministic (measured via trace2_data_intmax added in\npatch 4). Wall-clock times are medians over 10-20 runs with CPU governor set\nto performance.\n\n2.6M-commit monorepo with commit-graph (baseline v2.55.0-rc1):\n\n steps wall-clock\nmerge-base --all (across import) 2682391 -> 53521 7.26s -> 88ms\nmerge-base --all (1000 apart) 2659607 -> 1106 6.98s -> 8ms\nmerge-tree (across import) - 8.11s -> 100ms\n\n\ngit.git (88k commits, commit-graph):\n\n steps wall-clock\nmerge-base --all v2.0.0 v2.55.0-rc1 72264 -> 44589 82ms -> 49ms\nmerge-base --all HEAD HEAD~1000 9873 -> 3817 19ms -> 9ms\nmerge-base --all HEAD HEAD~10000 72285 -> 41523 80ms -> 48ms\nmerge-base HEAD HEAD~1000 - 9ms -> 9ms\nmerge-base --is-ancestor HEAD~1000 HEAD - 6ms -> 6ms\n\n\nChanges since v1:\n\n * Reordered patches: documentation first (describing the existing\n algorithm), tests before code changes, so they demonstrate passing with\n old logic first.\n\n * Dropped the ahead_behind decoupling patch. paint_state is now a NEW\n struct alongside nonstale_queue instead of replacing it. ahead_behind()\n is completely untouched.\n\n * Removed nonstale_queue_put_dedup() and nonstale_queue_get_dedup() (dead\n code after the conversion) in a separate commit.\n\n * Renamed: struct paint_queue -> paint_state, field pq -> queue,\n paint_count_add/remove -> paint_count_update (single function with signed\n delta parameter).\n\n * Split the old paint_count_transition (which handled both old and new\n flags in one call) into separate remove/add calls with a signed delta.\n This eliminates the need for the case 0 handler (which tracked \"not in\n the queue\") and allows an exhaustive switch on (PARENT1 | PARENT2 |\n STALE) that documents all valid flag combinations, with BUG() in default.\n\n * Moved all termination conditions into paint_queue_get(). The all-zero\n check and the side-exhaustion check are merged under a shared\n !pending_merge_bases guard. paint_queue_get() derives the generation from\n the dequeued commit itself, so no extra parameter is needed.\n\n * Added trace2_data_intmax() instrumentation to report the number of\n commits visited per paint walk (separate commit), with deterministic\n step-count assertions in t6600.\n\n * Expanded switch statements to multi-line format per .clang-format.\n\n * Used !count style throughout instead of count == 0.\n\n * Updated technical documentation alongside code changes.\n\n * Added benchmark data (both git-bench wall-clock and trace2 step counts)\n to commit messages.\n\n[1]\nhttps://lore.kernel.org/git/CAL71e4Ps-2_0+uuZu43N9pFnXBemoAohPs_eyRJf8taXHJPAXQ@mail.gmail.com/T/#u\n[2] https://github.com/gitgitgadget/git/pull/2150\n\nElijah Newren (1):\n t6600: add test cases for side-exhaustion edge cases\n\nKristofer Karlsson (6):\n Documentation/technical: add paint-down-to-common doc\n t6099, t6600: add side-exhaustion regression tests\n commit-reach: add trace2 instrumentation to paint_down_to_common()\n commit-reach: introduce struct paint_state with per-side counters\n commit-reach: remove unused nonstale_queue dedup wrappers\n commit-reach: terminate merge-base walk when one paint side is\n exhausted\n\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 128 ++++++++++++++\n commit-reach.c | 119 ++++++++++---\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 +++++++++\n t/t6600-test-reach.sh | 157 ++++++++++++++++++\n 7 files changed, 464 insertions(+), 25 deletions(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\n\nbase-commit: ab776a62a78576513ee121424adb19597fbb7613\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2149%2Fspkrka%2Fside-exhaust-pr-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2149/spkrka/side-exhaust-pr-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/2149\n\nRange-diff vs v1:\n\n 1: 5492acda0a < -: ---------- commit-reach: decouple ahead_behind from nonstale_queue\n 6: 9cbfc67d72 ! 1: 19ed743bd1 Documentation/technical: add paint-down-to-common doc\n @@ Commit message\n Documentation/technical: add paint-down-to-common doc\n \n Add a technical document describing the paint_down_to_common()\n - algorithm used for merge-base computation.\n + algorithm used for merge-base computation, covering the paint\n + walk, generation number regions, and termination conditions.\n \n Signed-off-by: Kristofer Karlsson \n \n @@ Documentation/technical/paint-down-to-common.adoc (new)\n +Termination\n +-----------\n +\n -+Termination happens when we can prove that no extra progress is\n -+possible. We are done with the main loop when one of the following\n -+conditions holds:\n ++The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n ++tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n ++so far. Once that commit is dequeued, every remaining entry is known\n ++to be STALE and the loop terminates. Specifically, the main loop\n ++ends when one of the following conditions holds:\n +\n + 1. The queue is empty.\n -+ 2. The queue only contains STALE entries.\n -+ 3. Side-exhaustion: the walk has reached the finite region and one\n -+ of the sides is fully exhausted.\n -+\n -+The loop waits for all pending merge-base candidates to be popped\n -+and recorded before any early exit fires, so no separate drain phase\n -+is needed after termination.\n ++ 2. `max_nonstale` has been dequeued, meaning the queue only contains\n ++ STALE entries.\n +\n +Stale entry condition\n +~~~~~~~~~~~~~~~~~~~~~\n -+If all entries are stale we cannot find any new merge bases since\n -+that requires at least one enqueued side node meeting the other side.\n -+However, we could still invalidate merge bases (if there are more\n -+than one). This is unnecessary since `remove_redundant()` will clean\n -+that up as a post-process step.\n -+\n -+Side-exhaustion\n -+~~~~~~~~~~~~~~~\n -+A commit is *exclusive* to one side if it carries that side's paint\n -+but not the other (e.g. PARENT1 without PARENT2).\n -+\n -+If we have reached the finite region of the graph, no future\n -+traversal step can add paint to an already-visited commit. Thus if\n -+there are no exclusive PARENT2 commits in the queue, no additional\n -+PARENT2 paint can be introduced into the walk. Even if exclusive\n -+PARENT1 commits remain, no new merge-base candidates can be\n -+discovered. The same holds symmetrically for PARENT1.\n -+\n -+This invariant is only valid in the finite region of the graph.\n ++Once all queued entries are stale, no new merge-base candidates can\n ++be discovered -- that requires at least one non-stale commit from\n ++each side meeting. Continuing the walk could still invalidate\n ++existing candidates by proving one is an ancestor of another, but\n ++`remove_redundant()` handles that as a post-processing step, so it\n ++is safe to exit early.\n +\n +Related documentation\n +---------------------\n +\n + - `Documentation/technical/commit-graph.adoc` -- generation numbers\n + and the reachability closure property.\n +\n + ## commit-reach.c ##\n +@@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n + \treturn commit;\n + }\n + \n +-/* all input commits in one and twos[] must have been parsed! */\n ++/*\n ++ * See Documentation/technical/paint-down-to-common.adoc\n ++ *\n ++ * All input commits in one and twos[] must have been parsed!\n ++ */\n + static int paint_down_to_common(struct repository *r,\n + \t\t\t\tstruct commit *one, int n,\n + \t\t\t\tstruct commit **twos,\n 4: 91372b975f ! 2: 6151b8e0a3 t6600: add test cases for side-exhaustion edge cases\n @@ t/t6600-test-reach.sh: test_expect_success 'setup' '\n +\t# ps-T1 ps-T2\n +\t#\n +\t# where ps-T1=merge(ps-Z,ps-B), ps-T2=merge(ps-W,ps-B), so\n -+\t# merge-base(ps-T1,ps-T2) = ps-B. During the walk, ps-X transitions\n ++\t# merge-base(ps-T1,ps-T2) = ps-B. During the walk, ps-X transitions\n +\t# to (PARENT1|PARENT2) via ps-Z and ps-W before ps-B is dequeued;\n +\t# then the STALE-walk from ps-B transitions ps-X to\n +\t# (PARENT1|PARENT2|STALE).\n @@ t/t6600-test-reach.sh: test_expect_success 'setup' '\n +\n +\t# Build a side topology that lives entirely outside the half\n +\t# commit-graph and has non-monotonic commit dates, to exercise the\n -+\t# INFINITY-gate in paint_down_to_common. With both tips outside\n ++\t# INFINITY-gate in paint_down_to_common. With both tips outside\n +\t# the graph, generation is INFINITY and the queue falls back to\n +\t# commit-date order, which here is non-monotonic.\n +\t#\n @@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many' '\n +\n +test_expect_success 'get_merge_bases_many:pending-stale' '\n +\t# Exercises the (PARENT1|PARENT2) -> (...|STALE) transition path in\n -+\t# paint_down_to_common(). See the topology comment in the setup test.\n ++\t# paint_down_to_common(). See the topology comment in the setup test.\n +\tcat >input <<-\\EOF &&\n +\tA:ps-T1\n +\tX:ps-T2\n @@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many' '\n +'\n +\n +test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n -+\t# Exercises the push-time INFINITY-gate in paint_down_to_common(). See\n ++\t# Exercises the push-time INFINITY-gate in paint_down_to_common(). See\n +\t# the pi-* topology comment in the setup test.\n +\tcat >input <<-\\EOF &&\n +\tA:pi-X\n 5: faf5bc98ed ! 3: 90f09ecb5c t6099, t6600: add side-exhaustion regression tests\n @@ t/t6099-merge-base-side-exhaustion.sh (new)\n +\n +Test that merge-base --all correctly handles cases where\n +multiple merge-base candidates exist and one is an ancestor\n -+of another. The side-exhaustion optimization in\n ++of another. The side-exhaustion optimization in\n +paint_down_to_common may exit before STALE propagation\n +removes the ancestor, but remove_redundant catches it.\n +\n @@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:infinity-both-s\n +test_expect_success 'setup mixed finite/INFINITY topology' '\n +\t# Create a commit outside all saved commit-graph files so it always\n +\t# has INFINITY generation, while its parent (ps-X) is in the graph\n -+\t# with a finite generation. Use the ps-* orphan topology so we do\n ++\t# with a finite generation. Use the ps-* orphan topology so we do\n +\t# not pollute the grid-based rev-list tests.\n +\tgit checkout ps-X &&\n +\ttest_env GIT_TEST_COMMIT_GRAPH= test_commit pm-INF\n @@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:infinity-both-s\n +test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n +\t# One tip (pm-INF) is outside the commit-graph with INFINITY\n +\t# generation; the other (ps-B) is in the graph with finite\n -+\t# generation. The walk starts in the INFINITY region and crosses\n ++\t# generation. The walk starts in the INFINITY region and crosses\n +\t# into the finite region where side-exhaustion can fire.\n +\tcat >input <<-\\EOF &&\n +\tA:pm-INF\n -: ---------- > 4: 6ade4df2ed commit-reach: add trace2 instrumentation to paint_down_to_common()\n 2: 316e4dfe26 ! 5: f24edd45f0 commit-reach: introduce struct paint_queue with per-side counters\n @@ Metadata\n Author: Kristofer Karlsson \n \n ## Commit message ##\n - commit-reach: introduce struct paint_queue with per-side counters\n + commit-reach: introduce struct paint_state with per-side counters\n \n - Replace the nonstale_queue abstraction in paint_down_to_common() with\n - a new paint_queue struct that tracks per-side commit counts. Each\n - non-stale queued commit occupies exactly one counter bucket based on\n - its paint flags: PARENT1-only, PARENT2-only, or both sides (a pending\n + Add a paint_state struct for use by paint_down_to_common() that\n + wraps a prio_queue with per-side commit counters. Each non-stale\n + queued commit occupies exactly one counter bucket based on its\n + paint flags: PARENT1-only, PARENT2-only, or both sides (a pending\n merge-base candidate).\n \n - The counters are maintained by paint_count_transition() which handles\n - all flag changes as bucket transfers: remove from the old bucket, add\n - to the new one. Either step is a no-op when the respective state has\n - no bucket (stale or zero).\n + The counters are maintained by paint_count_update() which adjusts\n + the appropriate bucket by a signed delta. An exhaustive switch on\n + the paint+stale bits documents all valid flag combinations in one\n + place.\n \n - The loop now drains the queue via paint_queue_get() and breaks when\n - all counters reach zero, replacing the old pointer-based termination\n - (max_nonstale). This is equivalent behavior.\n + Convert paint_down_to_common() to use paint_state. The loop now\n + drains the queue via paint_queue_get() which returns NULL when all\n + counters reach zero, replacing the old pointer-based termination\n + (max_nonstale). This is equivalent behavior -- both conditions\n + detect that no non-stale entries remain.\n +\n + The existing nonstale_queue is left in place for ahead_behind().\n +\n + Step counts (via trace2 from the previous commit) are identical\n + before and after this refactoring, confirming no behavioral change.\n \n Signed-off-by: Kristofer Karlsson \n \n + ## Documentation/technical/paint-down-to-common.adoc ##\n +@@ Documentation/technical/paint-down-to-common.adoc: re-enqueued is bounded by the number of flag transitions.\n + Termination\n + -----------\n + \n +-The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n +-tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n +-so far. Once that commit is dequeued, every remaining entry is known\n +-to be STALE and the loop terminates. Specifically, the main loop\n ++The walk tracks the number of commits of each type in the queue\n ++(PARENT1-only, PARENT2-only, pending merge-base). The main loop\n + ends when one of the following conditions holds:\n + \n + 1. The queue is empty.\n +- 2. `max_nonstale` has been dequeued, meaning the queue only contains\n +- STALE entries.\n ++ 2. The queue contains only stale entries.\n + \n + Stale entry condition\n + ~~~~~~~~~~~~~~~~~~~~~\n +\n ## commit-reach.c ##\n -@@ commit-reach.c: static int compare_commits_by_gen(const void *_a, const void *_b)\n +@@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n + \treturn commit;\n }\n \n - /*\n -- * A prio_queue with O(1) termination check. 'max_nonstale' tracks\n -- * the lowest-priority non-stale commit enqueued so far; once it is\n -- * popped, every remaining entry is known to be STALE.\n ++/*\n + * Priority queue with per-side commit counters for paint_down_to_common().\n + * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n + * PARENT2-only, or both (a pending merge-base candidate).\n - */\n --struct nonstale_queue {\n -+struct paint_queue {\n - \tstruct prio_queue pq;\n --\tstruct commit *max_nonstale;\n ++ */\n ++struct paint_state {\n ++\tstruct prio_queue queue;\n +\tint p1_count;\n +\tint p2_count;\n +\tint pending_merge_bases;\n - };\n - \n --static void nonstale_queue_put(struct nonstale_queue *queue,\n --\t\t\t struct commit *c)\n -+/*\n -+ * Adjust per-side counters for a paint-state transition. Non-stale\n -+ * commits are counted in one of three counters: PARENT1-only,\n -+ * PARENT2-only, or both. Zero means \"not in the queue\" (used on\n -+ * enqueue/dequeue); stale commits are not counted at all.\n -+ */\n -+static void paint_count_transition(struct paint_queue *queue,\n -+\t\t\t\t unsigned old_flags, unsigned new_flags)\n - {\n --\tstruct commit *old = queue->max_nonstale;\n -+\tunsigned old_paint = old_flags & (PARENT1 | PARENT2 | STALE);\n -+\tunsigned new_paint = new_flags & (PARENT1 | PARENT2 | STALE);\n - \n --\tprio_queue_put(&queue->pq, c);\n --\tif (c->object.flags & STALE)\n -+\tif (old_paint == new_paint)\n - \t\treturn;\n --\tif (!old || queue->pq.compare(old, c, queue->pq.cb_data) <= 0)\n --\t\tqueue->max_nonstale = c;\n --}\n --\n --static struct commit *nonstale_queue_get(struct nonstale_queue *queue)\n --{\n --\tstruct commit *commit = prio_queue_get(&queue->pq);\n - \n --\tif (commit == queue->max_nonstale)\n --\t\tqueue->max_nonstale = NULL;\n --\n --\treturn commit;\n -+\tif (!(old_paint & STALE)) {\n -+\t\tswitch (old_paint & (PARENT1 | PARENT2)) {\n -+\t\tcase 0: break;\n -+\t\tcase PARENT1: queue->p1_count--; break;\n -+\t\tcase PARENT2: queue->p2_count--; break;\n -+\t\tcase PARENT1 | PARENT2: queue->pending_merge_bases--; break;\n -+\t\tdefault: BUG(\"unexpected paint state\");\n -+\t\t}\n -+\t}\n -+\tif (!(new_paint & STALE)) {\n -+\t\tswitch (new_paint & (PARENT1 | PARENT2)) {\n -+\t\tcase 0: break;\n -+\t\tcase PARENT1: queue->p1_count++; break;\n -+\t\tcase PARENT2: queue->p2_count++; break;\n -+\t\tcase PARENT1 | PARENT2: queue->pending_merge_bases++; break;\n -+\t\tdefault: BUG(\"unexpected paint state\");\n -+\t\t}\n ++};\n ++\n ++static void paint_count_update(struct paint_state *state,\n ++\t\t\t unsigned flags, int delta)\n ++{\n ++\tswitch (flags & (PARENT1 | PARENT2 | STALE)) {\n ++\tcase PARENT1:\n ++\t\tstate->p1_count += delta;\n ++\t\tbreak;\n ++\n ++\tcase PARENT2:\n ++\t\tstate->p2_count += delta;\n ++\t\tbreak;\n ++\n ++\tcase PARENT1 | PARENT2:\n ++\t\tstate->pending_merge_bases += delta;\n ++\t\tbreak;\n ++\n ++\tcase PARENT1 | PARENT2 | STALE:\n ++\t\tbreak;\n ++\n ++\tdefault:\n ++\t\tBUG(\"unexpected paint state\");\n +\t}\n - }\n - \n --static void clear_nonstale_queue(struct nonstale_queue *queue)\n -+static void paint_queue_put(struct paint_queue *queue,\n ++}\n ++\n ++static void paint_queue_put(struct paint_state *state,\n +\t\t\t struct commit *c, unsigned add_flags)\n - {\n --\tclear_prio_queue(&queue->pq);\n --\tqueue->max_nonstale = NULL;\n --}\n ++{\n +\tunsigned old_flags = c->object.flags;\n +\tc->object.flags |= add_flags;\n - \n --static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n --\t\t\t\t struct commit *c)\n --{\n --\tif (c->object.flags & ENQUEUED)\n --\t\treturn;\n --\tc->object.flags |= ENQUEUED;\n --\tnonstale_queue_put(queue, c);\n ++\n +\tif (old_flags & ENQUEUED) {\n -+\t\tpaint_count_transition(queue, old_flags, c->object.flags);\n ++\t\tpaint_count_update(state, old_flags, -1);\n ++\t\tpaint_count_update(state, c->object.flags, 1);\n +\t} else {\n +\t\tc->object.flags |= ENQUEUED;\n -+\t\tprio_queue_put(&queue->pq, c);\n -+\t\tpaint_count_transition(queue, 0, c->object.flags);\n ++\t\tprio_queue_put(&state->queue, c);\n ++\t\tpaint_count_update(state, c->object.flags, 1);\n +\t}\n - }\n - \n --static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n -+static struct commit *paint_queue_get(struct paint_queue *queue)\n - {\n --\tstruct commit *commit = nonstale_queue_get(queue);\n -+\tstruct commit *commit = prio_queue_get(&queue->pq);\n - \n --\tif (commit)\n ++}\n ++\n ++static struct commit *paint_queue_get(struct paint_state *state)\n ++{\n ++\tstruct commit *commit;\n ++\n ++\tif (!state->p1_count && !state->p2_count &&\n ++\t !state->pending_merge_bases)\n ++\t\treturn NULL;\n ++\n ++\tcommit = prio_queue_get(&state->queue);\n +\tif (commit) {\n - \t\tcommit->object.flags &= ~ENQUEUED;\n -+\t\tpaint_count_transition(queue, commit->object.flags, 0);\n ++\t\tcommit->object.flags &= ~ENQUEUED;\n ++\t\tpaint_count_update(state, commit->object.flags, -1);\n +\t}\n - \treturn commit;\n - }\n - \n ++\treturn commit;\n ++}\n ++\n + /*\n + * See Documentation/technical/paint-down-to-common.adoc\n + *\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n \t\t\t\tenum merge_base_flags mb_flags,\n \t\t\t\tstruct commit_list **result)\n {\n -\tstruct nonstale_queue queue = {\n -\t\t{ compare_commits_by_gen_then_commit_date }\n -+\tstruct paint_queue queue = {\n -+\t\t.pq = { compare_commits_by_gen_then_commit_date }\n ++\tstruct paint_state state = {\n ++\t\t.queue = { compare_commits_by_gen_then_commit_date }\n \t};\n +\tstruct commit *commit;\n \tint i;\n + \tint steps = 0;\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n -@@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n + \n + \tif (!min_generation && !corrected_commit_dates_enabled(r))\n +-\t\tqueue.pq.compare = compare_commits_by_commit_date;\n ++\t\tstate.queue.compare = compare_commits_by_commit_date;\n + \n + \tone->object.flags |= PARENT1;\n + \tif (!n) {\n \t\tcommit_list_append(one, result);\n \t\treturn 0;\n \t}\n -\tnonstale_queue_put_dedup(&queue, one);\n -+\tpaint_queue_put(&queue, one, 0);\n ++\tpaint_queue_put(&state, one, 0);\n \n -\tfor (i = 0; i < n; i++) {\n -\t\ttwos[i]->object.flags |= PARENT2;\n -\t\tnonstale_queue_put_dedup(&queue, twos[i]);\n -\t}\n +\tfor (i = 0; i < n; i++)\n -+\t\tpaint_queue_put(&queue, twos[i], PARENT2);\n ++\t\tpaint_queue_put(&state, twos[i], PARENT2);\n \n -\twhile (queue.max_nonstale) {\n -\t\tstruct commit *commit = nonstale_queue_get_dedup(&queue);\n -+\twhile ((commit = paint_queue_get(&queue))) {\n ++\twhile ((commit = paint_queue_get(&state))) {\n \t\tstruct commit_list *parents;\n \t\tint flags;\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n \t\t\t\tcontinue;\n \t\t\tif (repo_parse_commit(r, p)) {\n -\t\t\t\tclear_nonstale_queue(&queue);\n -+\t\t\t\tclear_prio_queue(&queue.pq);\n ++\t\t\t\tclear_prio_queue(&state.queue);\n \t\t\t\tcommit_list_free(*result);\n \t\t\t\t*result = NULL;\n \t\t\t\t/*\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n \t\t\t}\n -\t\t\tp->object.flags |= flags;\n -\t\t\tnonstale_queue_put_dedup(&queue, p);\n -+\t\t\tpaint_queue_put(&queue, p, flags);\n ++\t\t\tpaint_queue_put(&state, p, flags);\n \t\t}\n -+\n -+\t\tif (queue.p1_count + queue.p2_count +\n -+\t\t queue.pending_merge_bases == 0)\n -+\t\t\tbreak;\n \t}\n \n -\tclear_nonstale_queue(&queue);\n -+\tclear_prio_queue(&queue.pq);\n ++\tclear_prio_queue(&state.queue);\n + \ttrace2_data_intmax(\"paint_down_to_common\", r,\n + \t\t\t \"steps\", steps);\n \tcommit_list_sort_by_date(result);\n - \treturn 0;\n - }\n -: ---------- > 6: 8c72f01083 commit-reach: remove unused nonstale_queue dedup wrappers\n 3: ed12a5cb5b ! 7: d84b932e5b commit-reach: terminate merge-base walk when one paint side is exhausted\n @@ Commit message\n commit-reach: terminate merge-base walk when one paint side is exhausted\n \n Add an early termination check to paint_down_to_common() using the\n - per-side counters introduced in the previous commit. Once the walk\n - enters the finite-generation region, terminate early when one side's\n - exclusive count drops to zero -- no new merge-base can form without\n - both paint sides meeting.\n + per-side counters introduced earlier. Once the walk enters the\n + finite-generation region, terminate early when one side's exclusive\n + count drops to zero -- no new merge-base can form without both paint\n + sides meeting.\n \n The check also waits for pending_merge_bases to reach zero, ensuring\n - all merge-base candidates have been popped and recorded before\n + all merge-base candidates have been dequeued and recorded before\n exiting.\n \n The INFINITY gate ensures correctness: commits without a commit-graph\n @@ Commit message\n once the walk enters the finite-generation region where ordering\n guarantees hold.\n \n - On large repositories with commit-graph, this yields 100-1000x\n - speedups for merge-base queries where one side (e.g. a PR branch) is\n - much smaller than the other.\n + Step counts measured with trace2 on git.git with commit-graph:\n +\n + merge-base --all v2.0.0 v2.55.0-rc1:\n + before: 72264 steps after: 44589 steps\n +\n + merge-base --all v2.55.0-rc1 v2.55.0-rc1~5:\n + before: 110 steps after: 7 steps\n \n Helped-by: Derrick Stolee \n Helped-by: Elijah Newren \n Signed-off-by: Kristofer Karlsson \n \n + ## Documentation/technical/paint-down-to-common.adoc ##\n +@@ Documentation/technical/paint-down-to-common.adoc: ends when one of the following conditions holds:\n + \n + 1. The queue is empty.\n + 2. The queue contains only stale entries.\n ++ 3. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n ++ remain in the queue, no pending merge-base candidates exist,\n ++ and the walk has entered the finite-generation region.\n + \n + Stale entry condition\n + ~~~~~~~~~~~~~~~~~~~~~\n +@@ Documentation/technical/paint-down-to-common.adoc: existing candidates by proving one is an ancestor of another, but\n + `remove_redundant()` handles that as a post-processing step, so it\n + is safe to exit early.\n + \n ++Side-exhaustion condition\n ++~~~~~~~~~~~~~~~~~~~~~~~~~\n ++A new merge-base requires commits from both sides to meet. When one\n ++side's exclusive counter reaches zero and there are no pending\n ++merge-base candidates, no future traversal step can produce a new\n ++candidate.\n ++\n ++This optimization only activates in the finite-generation region\n ++where topological ordering holds. In that region, children are\n ++always visited before parents, so paint flags are final at visit\n ++time and an exhausted side cannot reappear. In the INFINITY region,\n ++commit-date ordering can violate this guarantee, so the check is\n ++skipped.\n ++\n + Related documentation\n + ---------------------\n + \n +\n ## commit-reach.c ##\n -@@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n - \t\tif (queue.p1_count + queue.p2_count +\n - \t\t queue.pending_merge_bases == 0)\n - \t\t\tbreak;\n +@@ commit-reach.c: static void paint_queue_put(struct paint_state *state,\n + \n + static struct commit *paint_queue_get(struct paint_state *state)\n + {\n +-\tstruct commit *commit;\n ++\tstruct commit *commit = prio_queue_get(&state->queue);\n + \n +-\tif (!state->p1_count && !state->p2_count &&\n +-\t !state->pending_merge_bases)\n ++\tif (!commit)\n + \t\treturn NULL;\n + \n +-\tcommit = prio_queue_get(&state->queue);\n +-\tif (commit) {\n +-\t\tcommit->object.flags &= ~ENQUEUED;\n +-\t\tpaint_count_update(state, commit->object.flags, -1);\n ++\tcommit->object.flags &= ~ENQUEUED;\n +\n ++\tif (!state->pending_merge_bases) {\n ++\t\tif (!state->p1_count && !state->p2_count)\n ++\t\t\treturn NULL;\n +\t\t/*\n +\t\t * Side exhaustion: a new merge-base can only form\n +\t\t * when both PARENT1-only and PARENT2-only commits\n -+\t\t * remain in the queue. In the finite-generation\n ++\t\t * remain in the queue. In the finite-generation\n +\t\t * region the queue is ordered topologically, so\n +\t\t * no future step can add paint to visited commits\n +\t\t * and an exhausted side cannot reappear.\n +\t\t */\n -+\t\tif (generation < GENERATION_NUMBER_INFINITY &&\n -+\t\t queue.pending_merge_bases == 0 &&\n -+\t\t (queue.p1_count == 0 || queue.p2_count == 0))\n -+\t\t\tbreak;\n ++\t\tif ((!state->p1_count || !state->p2_count) &&\n ++\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n ++\t\t\treturn NULL;\n \t}\n ++\n ++\tpaint_count_update(state, commit->object.flags, -1);\n + \treturn commit;\n + }\n + \n +\n + ## t/t6600-test-reach.sh ##\n +@@ t/t6600-test-reach.sh: test_expect_success 'merge-base --all commit-walk steps' '\n + \tcp commit-graph-full .git/objects/info/commit-graph &&\n + \tGIT_TRACE2_EVENT=\"$(pwd)/trace-full.txt\" \\\n + \t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n +-\ttest_trace2_data paint_down_to_common steps 80 actual &&\n +-\ttest_trace2_data paint_down_to_common steps 81 \n\nAdd a technical document describing the paint_down_to_common()\nalgorithm used for merge-base computation, covering the paint\nwalk, generation number regions, and termination conditions.\n\nSigned-off-by: Kristofer Karlsson \n---\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 114 ++++++++++++++++++\n commit-reach.c | 6 +-\n 4 files changed, 121 insertions(+), 1 deletion(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 2699f0b24a..f8dea4b395 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -129,6 +129,7 @@ TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\n TECH_DOCS += technical/packfile-uri\n TECH_DOCS += technical/pack-heuristics\n+TECH_DOCS += technical/paint-down-to-common\n TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/platform-support\ndiff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\nindex ec07088c57..9ce11d5e48 100644\n--- a/Documentation/technical/meson.build\n+++ b/Documentation/technical/meson.build\n@@ -18,6 +18,7 @@ articles = [\n 'multi-pack-index.adoc',\n 'packfile-uri.adoc',\n 'pack-heuristics.adoc',\n+ 'paint-down-to-common.adoc',\n 'parallel-checkout.adoc',\n 'partial-clone.adoc',\n 'platform-support.adoc',\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nnew file mode 100644\nindex 0000000000..c10d5d2887\n--- /dev/null\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -0,0 +1,114 @@\n+Merge-Base Computation and paint_down_to_common()\n+==================================================\n+\n+The function `paint_down_to_common()` in `commit-reach.c` computes merge\n+bases by walking the commit graph backwards from two sets of tips and\n+finding where their ancestry meets.\n+\n+Use cases\n+---------\n+\n+Computing merge bases is used in two different ways:\n+\n+ 1. *Finding all merge bases* (`merge-base --all`, `merge-tree`,\n+ `merge`, `rebase`). A merge base is a common ancestor that is\n+ not itself an ancestor of another common ancestor.\n+\n+ 2. *Ancestry checks* (`in_merge_bases`, used by `merge-base\n+ --is-ancestor`, `branch -d`, `fetch`). These ask: \"is commit A\n+ an ancestor of commit B?\" If a common ancestor equals one of the\n+ inputs, that input is necessarily the only merge base -- no other\n+ common ancestor can be both as recent and not an ancestor of it.\n+\n+Both use cases share the same algorithm and implementation.\n+\n+Algorithm\n+---------\n+\n+Given a commit `one` and a set of commits `twos[]`, the walk paints\n+commits with two colors:\n+\n+ - PARENT1: reachable from `one`\n+ - PARENT2: reachable from any commit in `twos[]`\n+\n+The walk uses a priority queue ordered by generation number (falling\n+back to commit date when generation numbers are unavailable). Each\n+step dequeues the highest-priority commit (this is when we say a\n+commit is \"visited\") and propagates its paint flags to its parents,\n+enqueuing them if they gained new flags. When a commit receives\n+both PARENT1 and PARENT2, it is a merge-base candidate. A candidate\n+gains the STALE flag so its ancestors propagate staleness -- any\n+deeper common ancestor is necessarily redundant.\n+\n+INFINITY and finite generation regions\n+--------------------------------------\n+\n+The commit-graph stores a generation number for each commit. Commits\n+not in the commit-graph have generation `GENERATION_NUMBER_INFINITY`. The\n+graph is closed under reachability: if a commit is in the graph, all\n+its ancestors are too. This partitions the commit graph into two regions:\n+\n+....\n+ +---------------------------------------+\n+ | INFINITY region |\n+ | generation = INFINITY |\n+ | queue order: heuristic (commit date) |\n+ +---------------------------------------+\n+ |\n+ v\n+ +---------------------------------------+\n+ | Finite region |\n+ | generation = finite |\n+ | queue order: topological |\n+ +---------------------------------------+\n+....\n+\n+When the commit-graph is enabled, the INFINITY region is typically\n+very small -- it only contains commits added since the last\n+commit-graph refresh.\n+\n+All reachable INFINITY-generation commits are visited before any\n+finite-generation commit, because INFINITY is larger than any finite\n+value. Once the walk crosses into the finite region, it stays there.\n+\n+In the finite region, generation ordering guarantees topological\n+traversal: children are always visited before their parents. This\n+means that paint on already-visited commits is final -- no future\n+traversal step can add paint to them.\n+\n+In the INFINITY region, commit-date ordering can violate this: a\n+parent with a later date can be visited before a child with an earlier\n+date. Paint flags are therefore NOT final at visit time, and a\n+commit visited with only one side's paint may later gain the other.\n+\n+Paint flags are only added, never removed. Since each flag can be set\n+at most once per commit, the number of times a commit can be\n+re-enqueued is bounded by the number of flag transitions.\n+\n+Termination\n+-----------\n+\n+The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n+tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n+so far. Once that commit is dequeued, every remaining entry is known\n+to be STALE and the loop terminates. Specifically, the main loop\n+ends when one of the following conditions holds:\n+\n+ 1. The queue is empty.\n+ 2. `max_nonstale` has been dequeued, meaning the queue only contains\n+ STALE entries.\n+\n+Stale entry condition\n+~~~~~~~~~~~~~~~~~~~~~\n+Once all queued entries are stale, no new merge-base candidates can\n+be discovered -- that requires at least one non-stale commit from\n+each side meeting. Continuing the walk could still invalidate\n+existing candidates by proving one is an ancestor of another, but\n+`remove_redundant()` handles that as a post-processing step, so it\n+is safe to exit early.\n+\n+Related documentation\n+---------------------\n+\n+ - `Documentation/technical/commit-graph.adoc` -- generation numbers\n+ and the reachability closure property.\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 5df471a313..a9483759e0 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -96,7 +96,11 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n \treturn commit;\n }\n \n-/* all input commits in one and twos[] must have been parsed! */\n+/*\n+ * See Documentation/technical/paint-down-to-common.adoc\n+ *\n+ * All input commits in one and twos[] must have been parsed!\n+ */\n static int paint_down_to_common(struct repository *r,\n \t\t\t\tstruct commit *one, int n,\n \t\t\t\tstruct commit **twos,\n-- \ngitgitgadget\n\n"},{"id":"546302","messageId":"6151b8e0a3989a51e6d9717e0ceac439f26f1c1d.1782303254.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","subject":"[PATCH v2 2/7] t6600: add test cases for side-exhaustion edge cases","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-24T12:14:08Z","receivedAt":"2026-06-24T12:14:20Z","isPatch":true,"body":"From: Elijah Newren \n\nAdd test cases to t6600-test-reach.sh that exercise edge cases in the\nside-exhaustion optimization for paint_down_to_common():\n\n - in_merge_bases_many:self: commit is both A and one of the X inputs\n - get_merge_bases_many:duplicate-twos: duplicate entries in X list\n - get_merge_bases_many:pending-stale: STALE transition on an\n already-painted commit (ps-* diamond topology)\n - get_merge_bases_many:infinity-both-sides: both tips outside the\n commit-graph with non-monotonic dates (pi-* topology)\n\nSigned-off-by: Elijah Newren \nSigned-off-by: Kristofer Karlsson \n---\n t/t6600-test-reach.sh | 111 ++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 111 insertions(+)\n\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex b5b314e570..c2e091aad1 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -49,6 +49,62 @@ test_expect_success 'setup' '\n \t\t\tgit tag -a -m \"$x-$i\" tag-$x-$i commit-$x-$i || return 1\n \t\tdone\n \tdone &&\n+\n+\t# Build a small side topology to exercise the (PARENT1|PARENT2) ->\n+\t# (PARENT1|PARENT2|STALE) transition in paint_down_to_common(); the\n+\t# 10x10 grid above does not exercise it because no merge-base candidate\n+\t# there is a descendant of another, so STALE never reaches a\n+\t# still-pending candidate.\n+\t#\n+\t# ps-X\n+\t# /|\\\n+\t# / | \\\n+\t# ps-Z ps-B ps-W\n+\t# | / \\ |\n+\t# | / \\ |\n+\t# |/ \\|\n+\t# ps-T1 ps-T2\n+\t#\n+\t# where ps-T1=merge(ps-Z,ps-B), ps-T2=merge(ps-W,ps-B), so\n+\t# merge-base(ps-T1,ps-T2) = ps-B. During the walk, ps-X transitions\n+\t# to (PARENT1|PARENT2) via ps-Z and ps-W before ps-B is dequeued;\n+\t# then the STALE-walk from ps-B transitions ps-X to\n+\t# (PARENT1|PARENT2|STALE).\n+\tgit checkout --orphan ps-orphan &&\n+\ttest_commit ps-X &&\n+\tgit checkout -b ps-B-br ps-X && test_commit ps-B &&\n+\tgit checkout -b ps-Z-br ps-X && test_commit ps-Z &&\n+\tgit checkout -b ps-W-br ps-X && test_commit ps-W &&\n+\tgit checkout -b ps-T1 ps-Z &&\n+\tgit merge --no-ff -m ps-T1 ps-B &&\n+\tgit checkout -b ps-T2 ps-W &&\n+\tgit merge --no-ff -m ps-T2 ps-B &&\n+\n+\t# Build a side topology that lives entirely outside the half\n+\t# commit-graph and has non-monotonic commit dates, to exercise the\n+\t# INFINITY-gate in paint_down_to_common. With both tips outside\n+\t# the graph, generation is INFINITY and the queue falls back to\n+\t# commit-date order, which here is non-monotonic.\n+\t#\n+\t# pi-X (date 500, PARENT1 tip) --> pi-P, pi-D\n+\t# pi-D (date 480) --> pi-C\n+\t# pi-C (date 200) --> pi-B\n+\t# pi-B (date 100, PARENT2 tip) --> pi-P\n+\t# pi-P (date 450, root)\n+\t#\n+\t# merge-base(pi-X, pi-B) = pi-B (it is an ancestor of pi-X and is\n+\t# itself one of the queried tips).\n+\tgit checkout --orphan pi-orphan &&\n+\ttest_commit --date \"@450 +0000\" pi-P &&\n+\ttest_commit --date \"@100 +0000\" pi-B &&\n+\ttest_commit --date \"@200 +0000\" pi-C &&\n+\ttest_commit --date \"@480 +0000\" pi-D &&\n+\tGIT_AUTHOR_DATE=\"@500 +0000\" GIT_COMMITTER_DATE=\"@500 +0000\" \\\n+\t\tgit commit-tree -p pi-D -p pi-P -m pi-X pi-D^{tree} >pi-X-oid &&\n+\tpi_x=\"$(cat pi-X-oid)\" &&\n+\tgit branch -f pi-X-br \"$pi_x\" &&\n+\tgit tag pi-X \"$pi_x\" &&\n+\n \tgit commit-graph write --reachable &&\n \tmv .git/objects/info/commit-graph commit-graph-full &&\n \tchmod u+w commit-graph-full &&\n@@ -146,6 +202,16 @@ test_expect_success 'in_merge_bases_many:miss-heuristic' '\n \ttest_all_modes in_merge_bases_many\n '\n \n+test_expect_success 'in_merge_bases_many:self' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-8\n+\tX:commit-5-9\n+\tX:commit-6-8\n+\tEOF\n+\techo \"in_merge_bases_many(A,X):1\" >expect &&\n+\ttest_all_modes in_merge_bases_many\n+'\n+\n test_expect_success 'is_descendant_of:hit' '\n \tcat >input <<-\\EOF &&\n \tA:commit-5-7\n@@ -183,6 +249,51 @@ test_expect_success 'get_merge_bases_many' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'get_merge_bases_many:duplicate-twos' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tX:commit-4-8\n+\tX:commit-4-8\n+\tX:commit-6-6\n+\tX:commit-6-6\n+\tX:commit-8-3\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse commit-5-6 \\\n+\t\t\t commit-4-7 | sort\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n+test_expect_success 'get_merge_bases_many:pending-stale' '\n+\t# Exercises the (PARENT1|PARENT2) -> (...|STALE) transition path in\n+\t# paint_down_to_common(). See the topology comment in the setup test.\n+\tcat >input <<-\\EOF &&\n+\tA:ps-T1\n+\tX:ps-T2\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse ps-B\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n+test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n+\t# Exercises the push-time INFINITY-gate in paint_down_to_common(). See\n+\t# the pi-* topology comment in the setup test.\n+\tcat >input <<-\\EOF &&\n+\tA:pi-X\n+\tX:pi-B\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse pi-B\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546303","messageId":"90f09ecb5cd2141621d8f203189b976a20f6128b.1782303254.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","subject":"[PATCH v2 3/7] t6099, t6600: add side-exhaustion regression tests","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-24T12:14:09Z","receivedAt":"2026-06-24T12:14:22Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd t6099 to test the case where multiple merge-base candidates exist\nand one is an ancestor of another. This exercises the side-exhaustion\noptimization in paint_down_to_common together with the\nremove_redundant safety net in get_merge_bases_many_0.\n\nAdd a mixed finite/INFINITY test to t6600 where one tip is outside\nthe commit-graph (INFINITY generation) and the other is inside.\nThis exercises the region transition: the walk starts in the\nINFINITY region where side-exhaustion is disabled, then crosses\ninto the finite region where it can fire.\n\nSigned-off-by: Kristofer Karlsson \n---\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 +++++++++++++++++++++++++++\n t/t6600-test-reach.sh | 25 ++++++++\n 3 files changed, 108 insertions(+)\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\ndiff --git a/t/meson.build b/t/meson.build\nindex 3219264fe7..ee6ebdffb9 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -786,6 +786,7 @@ integration_tests = [\n 't6041-bisect-submodule.sh',\n 't6050-replace.sh',\n 't6060-merge-index.sh',\n+ 't6099-merge-base-side-exhaustion.sh',\n 't6100-rev-list-in-order.sh',\n 't6101-rev-parse-parents.sh',\n 't6102-rev-list-unexpected-objects.sh',\ndiff --git a/t/t6099-merge-base-side-exhaustion.sh b/t/t6099-merge-base-side-exhaustion.sh\nnew file mode 100755\nindex 0000000000..4f1e0d50ef\n--- /dev/null\n+++ b/t/t6099-merge-base-side-exhaustion.sh\n@@ -0,0 +1,82 @@\n+#!/bin/sh\n+\n+test_description='merge-base with ancestor among merge-base candidates\n+\n+Test that merge-base --all correctly handles cases where\n+multiple merge-base candidates exist and one is an ancestor\n+of another. The side-exhaustion optimization in\n+paint_down_to_common may exit before STALE propagation\n+removes the ancestor, but remove_redundant catches it.\n+\n+Graph shape (parents are below children):\n+\n+ A ----------- X\n+ |\\ /|\n+ | B---------/ |\n+ | | |\n+ e2 \\ f2\n+ | | |\n+ e1 d1 f1\n+ \\ | /\n+ \\ | /\n+ \\| /\n+ C\n+\n+A and X are the two tips.\n+B and C are both reachable from A and X.\n+B reaches C through d1.\n+Only B should appear in merge-base --all output.\n+'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+TEST_PASSES_SANITIZE_LEAK=true\n+. ./test-lib.sh\n+\n+test_expect_success 'setup ancestor merge-base candidate' '\n+\ttest_commit C &&\n+\n+\tgit checkout -b d-chain HEAD &&\n+\ttest_commit d1 &&\n+\ttest_commit B &&\n+\n+\tgit checkout -b e-path C &&\n+\ttest_commit e1 &&\n+\ttest_commit e2 &&\n+\n+\tgit checkout -b f-path C &&\n+\ttest_commit f1 &&\n+\ttest_commit f2 &&\n+\n+\tgit checkout -b branch-A e-path &&\n+\ttest_merge A B &&\n+\n+\tgit checkout -b branch-X f-path &&\n+\ttest_merge X B &&\n+\n+\tgit commit-graph write --reachable\n+'\n+\n+test_expect_success 'merge-base --all excludes ancestor candidate' '\n+\tgit rev-parse B >expected &&\n+\tgit merge-base --all A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'merge-base (single) finds shallowest' '\n+\tgit rev-parse B >expected &&\n+\tgit merge-base A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+# Without commit-graph: generation numbers are INFINITY,\n+# side-exhaustion optimization does not fire.\n+test_expect_success 'merge-base --all without commit-graph' '\n+\trm -f .git/objects/info/commit-graph &&\n+\tgit rev-parse B >expected &&\n+\tgit merge-base --all A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex c2e091aad1..4b771b4c58 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -294,6 +294,31 @@ test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'setup mixed finite/INFINITY topology' '\n+\t# Create a commit outside all saved commit-graph files so it always\n+\t# has INFINITY generation, while its parent (ps-X) is in the graph\n+\t# with a finite generation. Use the ps-* orphan topology so we do\n+\t# not pollute the grid-based rev-list tests.\n+\tgit checkout ps-X &&\n+\ttest_env GIT_TEST_COMMIT_GRAPH= test_commit pm-INF\n+'\n+\n+test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n+\t# One tip (pm-INF) is outside the commit-graph with INFINITY\n+\t# generation; the other (ps-B) is in the graph with finite\n+\t# generation. The walk starts in the INFINITY region and crosses\n+\t# into the finite region where side-exhaustion can fire.\n+\tcat >input <<-\\EOF &&\n+\tA:pm-INF\n+\tX:ps-B\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse ps-X\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546304","messageId":"6ade4df2ed2a836a3b4c5400ab13e8247e36c029.1782303254.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","subject":"[PATCH v2 4/7] commit-reach: add trace2 instrumentation to paint_down_to_common()","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-24T12:14:10Z","receivedAt":"2026-06-24T12:14:24Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a step counter and trace2_data_intmax() call so that the number\nof commits visited during the paint walk is observable via\nGIT_TRACE2_PERF. This provides a way to measure the impact of\nfuture optimizations without relying on wall-clock benchmarks alone.\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 5 +++++\n t/t6600-test-reach.sh | 21 +++++++++++++++++++++\n 2 files changed, 26 insertions(+)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex a9483759e0..f6a438550b 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -11,6 +11,7 @@\n #include \"tag.h\"\n #include \"commit-reach.h\"\n #include \"ewah/ewok.h\"\n+#include \"trace2.h\"\n \n /* Remember to update object flag allocation in object.h */\n #define PARENT1\t\t(1u<<16)\n@@ -112,6 +113,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t{ compare_commits_by_gen_then_commit_date }\n \t};\n \tint i;\n+\tint steps = 0;\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n@@ -135,6 +137,7 @@ static int paint_down_to_common(struct repository *r,\n \t\tstruct commit_list *parents;\n \t\tint flags;\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n+\t\tsteps++;\n \n \t\tif (min_generation && generation > last_gen)\n \t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n@@ -190,6 +193,8 @@ static int paint_down_to_common(struct repository *r,\n \t}\n \n \tclear_nonstale_queue(&queue);\n+\ttrace2_data_intmax(\"paint_down_to_common\", r,\n+\t\t\t \"steps\", steps);\n \tcommit_list_sort_by_date(result);\n \treturn 0;\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 4b771b4c58..c1109fb42f 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -319,6 +319,27 @@ test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'merge-base --all commit-walk steps' '\n+\ttest_when_finished rm -rf .git/objects/info/commit-graph \\\n+\t\t.git/objects/info/commit-graphs &&\n+\trm -rf .git/objects/info/commit-graph \\\n+\t\t.git/objects/info/commit-graphs &&\n+\n+\tGIT_TRACE2_EVENT=\"$(pwd)/trace-none.txt\" \\\n+\t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n+\ttest_trace2_data paint_down_to_common steps 81 actual &&\n+\ttest_trace2_data paint_down_to_common steps 80 actual &&\n+\ttest_trace2_data paint_down_to_common steps 81 input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546305","messageId":"f24edd45f0af1da64513164d5d720fe70c1decff.1782303254.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","subject":"[PATCH v2 5/7] commit-reach: introduce struct paint_state with per-side counters","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-24T12:14:11Z","receivedAt":"2026-06-24T12:14:25Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a paint_state struct for use by paint_down_to_common() that\nwraps a prio_queue with per-side commit counters. Each non-stale\nqueued commit occupies exactly one counter bucket based on its\npaint flags: PARENT1-only, PARENT2-only, or both sides (a pending\nmerge-base candidate).\n\nThe counters are maintained by paint_count_update() which adjusts\nthe appropriate bucket by a signed delta. An exhaustive switch on\nthe paint+stale bits documents all valid flag combinations in one\nplace.\n\nConvert paint_down_to_common() to use paint_state. The loop now\ndrains the queue via paint_queue_get() which returns NULL when all\ncounters reach zero, replacing the old pointer-based termination\n(max_nonstale). This is equivalent behavior -- both conditions\ndetect that no non-stale entries remain.\n\nThe existing nonstale_queue is left in place for ahead_behind().\n\nStep counts (via trace2 from the previous commit) are identical\nbefore and after this refactoring, confirming no behavioral change.\n\nSigned-off-by: Kristofer Karlsson \n---\n .../technical/paint-down-to-common.adoc | 9 +-\n commit-reach.c | 93 ++++++++++++++++---\n 2 files changed, 82 insertions(+), 20 deletions(-)\n\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nindex c10d5d2887..0f4e1892a5 100644\n--- a/Documentation/technical/paint-down-to-common.adoc\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -88,15 +88,12 @@ re-enqueued is bounded by the number of flag transitions.\n Termination\n -----------\n \n-The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n-tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n-so far. Once that commit is dequeued, every remaining entry is known\n-to be STALE and the loop terminates. Specifically, the main loop\n+The walk tracks the number of commits of each type in the queue\n+(PARENT1-only, PARENT2-only, pending merge-base). The main loop\n ends when one of the following conditions holds:\n \n 1. The queue is empty.\n- 2. `max_nonstale` has been dequeued, meaning the queue only contains\n- STALE entries.\n+ 2. The queue contains only stale entries.\n \n Stale entry condition\n ~~~~~~~~~~~~~~~~~~~~~\ndiff --git a/commit-reach.c b/commit-reach.c\nindex f6a438550b..bf102f5e28 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -97,6 +97,74 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n \treturn commit;\n }\n \n+/*\n+ * Priority queue with per-side commit counters for paint_down_to_common().\n+ * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n+ * PARENT2-only, or both (a pending merge-base candidate).\n+ */\n+struct paint_state {\n+\tstruct prio_queue queue;\n+\tint p1_count;\n+\tint p2_count;\n+\tint pending_merge_bases;\n+};\n+\n+static void paint_count_update(struct paint_state *state,\n+\t\t\t unsigned flags, int delta)\n+{\n+\tswitch (flags & (PARENT1 | PARENT2 | STALE)) {\n+\tcase PARENT1:\n+\t\tstate->p1_count += delta;\n+\t\tbreak;\n+\n+\tcase PARENT2:\n+\t\tstate->p2_count += delta;\n+\t\tbreak;\n+\n+\tcase PARENT1 | PARENT2:\n+\t\tstate->pending_merge_bases += delta;\n+\t\tbreak;\n+\n+\tcase PARENT1 | PARENT2 | STALE:\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tBUG(\"unexpected paint state\");\n+\t}\n+}\n+\n+static void paint_queue_put(struct paint_state *state,\n+\t\t\t struct commit *c, unsigned add_flags)\n+{\n+\tunsigned old_flags = c->object.flags;\n+\tc->object.flags |= add_flags;\n+\n+\tif (old_flags & ENQUEUED) {\n+\t\tpaint_count_update(state, old_flags, -1);\n+\t\tpaint_count_update(state, c->object.flags, 1);\n+\t} else {\n+\t\tc->object.flags |= ENQUEUED;\n+\t\tprio_queue_put(&state->queue, c);\n+\t\tpaint_count_update(state, c->object.flags, 1);\n+\t}\n+}\n+\n+static struct commit *paint_queue_get(struct paint_state *state)\n+{\n+\tstruct commit *commit;\n+\n+\tif (!state->p1_count && !state->p2_count &&\n+\t !state->pending_merge_bases)\n+\t\treturn NULL;\n+\n+\tcommit = prio_queue_get(&state->queue);\n+\tif (commit) {\n+\t\tcommit->object.flags &= ~ENQUEUED;\n+\t\tpaint_count_update(state, commit->object.flags, -1);\n+\t}\n+\treturn commit;\n+}\n+\n /*\n * See Documentation/technical/paint-down-to-common.adoc\n *\n@@ -109,31 +177,29 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\tenum merge_base_flags mb_flags,\n \t\t\t\tstruct commit_list **result)\n {\n-\tstruct nonstale_queue queue = {\n-\t\t{ compare_commits_by_gen_then_commit_date }\n+\tstruct paint_state state = {\n+\t\t.queue = { compare_commits_by_gen_then_commit_date }\n \t};\n+\tstruct commit *commit;\n \tint i;\n \tint steps = 0;\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n \tif (!min_generation && !corrected_commit_dates_enabled(r))\n-\t\tqueue.pq.compare = compare_commits_by_commit_date;\n+\t\tstate.queue.compare = compare_commits_by_commit_date;\n \n \tone->object.flags |= PARENT1;\n \tif (!n) {\n \t\tcommit_list_append(one, result);\n \t\treturn 0;\n \t}\n-\tnonstale_queue_put_dedup(&queue, one);\n+\tpaint_queue_put(&state, one, 0);\n \n-\tfor (i = 0; i < n; i++) {\n-\t\ttwos[i]->object.flags |= PARENT2;\n-\t\tnonstale_queue_put_dedup(&queue, twos[i]);\n-\t}\n+\tfor (i = 0; i < n; i++)\n+\t\tpaint_queue_put(&state, twos[i], PARENT2);\n \n-\twhile (queue.max_nonstale) {\n-\t\tstruct commit *commit = nonstale_queue_get_dedup(&queue);\n+\twhile ((commit = paint_queue_get(&state))) {\n \t\tstruct commit_list *parents;\n \t\tint flags;\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n@@ -172,7 +238,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t\tif ((p->object.flags & flags) == flags)\n \t\t\t\tcontinue;\n \t\t\tif (repo_parse_commit(r, p)) {\n-\t\t\t\tclear_nonstale_queue(&queue);\n+\t\t\t\tclear_prio_queue(&state.queue);\n \t\t\t\tcommit_list_free(*result);\n \t\t\t\t*result = NULL;\n \t\t\t\t/*\n@@ -187,12 +253,11 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\treturn error(_(\"could not parse commit %s\"),\n \t\t\t\t\t oid_to_hex(&p->object.oid));\n \t\t\t}\n-\t\t\tp->object.flags |= flags;\n-\t\t\tnonstale_queue_put_dedup(&queue, p);\n+\t\t\tpaint_queue_put(&state, p, flags);\n \t\t}\n \t}\n \n-\tclear_nonstale_queue(&queue);\n+\tclear_prio_queue(&state.queue);\n \ttrace2_data_intmax(\"paint_down_to_common\", r,\n \t\t\t \"steps\", steps);\n \tcommit_list_sort_by_date(result);\n-- \ngitgitgadget\n\n"},{"id":"546306","messageId":"8c72f01083237c00397dd074beda8f854e882cbe.1782303254.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","subject":"[PATCH v2 6/7] commit-reach: remove unused nonstale_queue dedup wrappers","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-24T12:14:12Z","receivedAt":"2026-06-24T12:14:27Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nnonstale_queue_put_dedup() and nonstale_queue_get_dedup() became\nunused after the previous commit. The core nonstale_queue functions\nremain in use by ahead_behind().\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 18 ------------------\n 1 file changed, 18 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex bf102f5e28..e0d9874f99 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -79,24 +79,6 @@ static void clear_nonstale_queue(struct nonstale_queue *queue)\n \tqueue->max_nonstale = NULL;\n }\n \n-static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n-\t\t\t\t struct commit *c)\n-{\n-\tif (c->object.flags & ENQUEUED)\n-\t\treturn;\n-\tc->object.flags |= ENQUEUED;\n-\tnonstale_queue_put(queue, c);\n-}\n-\n-static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n-{\n-\tstruct commit *commit = nonstale_queue_get(queue);\n-\n-\tif (commit)\n-\t\tcommit->object.flags &= ~ENQUEUED;\n-\treturn commit;\n-}\n-\n /*\n * Priority queue with per-side commit counters for paint_down_to_common().\n * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n-- \ngitgitgadget\n\n"},{"id":"546307","messageId":"d84b932e5b078edc8255b6944ecb67fc1aa086b0.1782303254.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","subject":"[PATCH v2 7/7] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-24T12:14:13Z","receivedAt":"2026-06-24T12:14:29Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd an early termination check to paint_down_to_common() using the\nper-side counters introduced earlier. Once the walk enters the\nfinite-generation region, terminate early when one side's exclusive\ncount drops to zero -- no new merge-base can form without both paint\nsides meeting.\n\nThe check also waits for pending_merge_bases to reach zero, ensuring\nall merge-base candidates have been dequeued and recorded before\nexiting.\n\nThe INFINITY gate ensures correctness: commits without a commit-graph\nentry have GENERATION_NUMBER_INFINITY and are ordered by commit date,\nwhich is not topologically reliable. The optimization only fires\nonce the walk enters the finite-generation region where ordering\nguarantees hold.\n\nStep counts measured with trace2 on git.git with commit-graph:\n\n merge-base --all v2.0.0 v2.55.0-rc1:\n before: 72264 steps after: 44589 steps\n\n merge-base --all v2.55.0-rc1 v2.55.0-rc1~5:\n before: 110 steps after: 7 steps\n\nHelped-by: Derrick Stolee \nHelped-by: Elijah Newren \nSigned-off-by: Kristofer Karlsson \n---\n .../technical/paint-down-to-common.adoc | 17 ++++++++++++\n commit-reach.c | 27 ++++++++++++++-----\n t/t6600-test-reach.sh | 4 +--\n 3 files changed, 39 insertions(+), 9 deletions(-)\n\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nindex 0f4e1892a5..983dfcf233 100644\n--- a/Documentation/technical/paint-down-to-common.adoc\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -94,6 +94,9 @@ ends when one of the following conditions holds:\n \n 1. The queue is empty.\n 2. The queue contains only stale entries.\n+ 3. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n+ remain in the queue, no pending merge-base candidates exist,\n+ and the walk has entered the finite-generation region.\n \n Stale entry condition\n ~~~~~~~~~~~~~~~~~~~~~\n@@ -104,6 +107,20 @@ existing candidates by proving one is an ancestor of another, but\n `remove_redundant()` handles that as a post-processing step, so it\n is safe to exit early.\n \n+Side-exhaustion condition\n+~~~~~~~~~~~~~~~~~~~~~~~~~\n+A new merge-base requires commits from both sides to meet. When one\n+side's exclusive counter reaches zero and there are no pending\n+merge-base candidates, no future traversal step can produce a new\n+candidate.\n+\n+This optimization only activates in the finite-generation region\n+where topological ordering holds. In that region, children are\n+always visited before parents, so paint flags are final at visit\n+time and an exhausted side cannot reappear. In the INFINITY region,\n+commit-date ordering can violate this guarantee, so the check is\n+skipped.\n+\n Related documentation\n ---------------------\n \ndiff --git a/commit-reach.c b/commit-reach.c\nindex e0d9874f99..f79d0b64d6 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -133,17 +133,30 @@ static void paint_queue_put(struct paint_state *state,\n \n static struct commit *paint_queue_get(struct paint_state *state)\n {\n-\tstruct commit *commit;\n+\tstruct commit *commit = prio_queue_get(&state->queue);\n \n-\tif (!state->p1_count && !state->p2_count &&\n-\t !state->pending_merge_bases)\n+\tif (!commit)\n \t\treturn NULL;\n \n-\tcommit = prio_queue_get(&state->queue);\n-\tif (commit) {\n-\t\tcommit->object.flags &= ~ENQUEUED;\n-\t\tpaint_count_update(state, commit->object.flags, -1);\n+\tcommit->object.flags &= ~ENQUEUED;\n+\n+\tif (!state->pending_merge_bases) {\n+\t\tif (!state->p1_count && !state->p2_count)\n+\t\t\treturn NULL;\n+\t\t/*\n+\t\t * Side exhaustion: a new merge-base can only form\n+\t\t * when both PARENT1-only and PARENT2-only commits\n+\t\t * remain in the queue. In the finite-generation\n+\t\t * region the queue is ordered topologically, so\n+\t\t * no future step can add paint to visited commits\n+\t\t * and an exhausted side cannot reappear.\n+\t\t */\n+\t\tif ((!state->p1_count || !state->p2_count) &&\n+\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n+\t\t\treturn NULL;\n \t}\n+\n+\tpaint_count_update(state, commit->object.flags, -1);\n \treturn commit;\n }\n \ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex c1109fb42f..03175befb3 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -332,12 +332,12 @@ test_expect_success 'merge-base --all commit-walk steps' '\n \tcp commit-graph-full .git/objects/info/commit-graph &&\n \tGIT_TRACE2_EVENT=\"$(pwd)/trace-full.txt\" \\\n \t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n-\ttest_trace2_data paint_down_to_common steps 80 actual &&\n-\ttest_trace2_data paint_down_to_common steps 81 Benchmarks\n> \n> Step counts are deterministic (measured via trace2_data_intmax added in\n> patch 4). Wall-clock times are medians over 10-20 runs with CPU governor set\n> to performance.\n> \n> 2.6M-commit monorepo with commit-graph (baseline v2.55.0-rc1):\n> \n> steps wall-clock\n> merge-base --all (across import) 2682391 -> 53521 7.26s -> 88ms\n> merge-base --all (1000 apart) 2659607 -> 1106 6.98s -> 8ms\n> merge-tree (across import) - 8.11s -> 100ms\n> \n> \n> git.git (88k commits, commit-graph):\n> \n> steps wall-clock\n> merge-base --all v2.0.0 v2.55.0-rc1 72264 -> 44589 82ms -> 49ms\n> merge-base --all HEAD HEAD~1000 9873 -> 3817 19ms -> 9ms\n> merge-base --all HEAD HEAD~10000 72285 -> 41523 80ms -> 48ms\n> merge-base HEAD HEAD~1000 - 9ms -> 9ms\n> merge-base --is-ancestor HEAD~1000 HEAD - 6ms -> 6ms\n\nI like seeing these updates including the deterministic steps. Is there\na reason you don't include the step data for 'merge-tree (across import)'\nin your monorepo case? The wall-clock is substantial, so it's not like the\nlast two examples in git.git where there may not be any difference.\n\nThanks,\n-Stolee\n\n"},{"id":"546321","messageId":"560c91df-3c07-4c8f-9924-ef0cc7646e08@gmail.com","threadId":"65846","inReplyTo":"6ade4df2ed2a836a3b4c5400ab13e8247e36c029.1782303254.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 4/7] commit-reach: add trace2 instrumentation to paint_down_to_common()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-24T13:41:12Z","receivedAt":"2026-06-24T13:41:14Z","isPatch":true,"body":"On 6/24/2026 8:14 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n> \n> Add a step counter and trace2_data_intmax() call so that the number\n> of commits visited during the paint walk is observable via\n> GIT_TRACE2_PERF. This provides a way to measure the impact of\n> future optimizations without relying on wall-clock benchmarks alone.\n\n> +\ttrace2_data_intmax(\"paint_down_to_common\", r,\n> +\t\t\t \"steps\", steps);\n\nThis is great data. Very clearly marked for what we should be\ndoing here.\n\n> +test_expect_success 'merge-base --all commit-walk steps' '\n> +\ttest_when_finished rm -rf .git/objects/info/commit-graph \\\n> +\t\t.git/objects/info/commit-graphs &&\n\n(highlighting this chunk)\n\n> +\trm -rf .git/objects/info/commit-graph \\\n> +\t\t.git/objects/info/commit-graphs &&\n> +\n> +\tGIT_TRACE2_EVENT=\"$(pwd)/trace-none.txt\" \\\n> +\t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n> +\ttest_trace2_data paint_down_to_common steps 81 +\n> +\tcp commit-graph-full .git/objects/info/commit-graph &&\n> +\tGIT_TRACE2_EVENT=\"$(pwd)/trace-full.txt\" \\\n> +\t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n> +\ttest_trace2_data paint_down_to_common steps 80 +\n> +\tcp commit-graph-half .git/objects/info/commit-graph &&\n> +\tGIT_TRACE2_EVENT=\"$(pwd)/trace-half.txt\" \\\n> +\t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n> +\ttest_trace2_data paint_down_to_common steps 81 +'\n> +\n\nThis test is a great example. I look forward to seeing that it\nupdates in the future.\n\nOne thing I was hoping to see was that your side-exhaustion tests\n(from patch v2 2/7) would also include these checks so they are\nmore obviously updating when the implementation updates later.\n\nOne way to accomplish that is to reorder this patch before adding\nthose tests so their first version includes these checks and then\nthe values update when changing the implementation.\n\nThanks,\n-Stolee\n\n\n"},{"id":"546322","messageId":"b4b33635-1279-46c0-819a-d29cc13921f5@gmail.com","threadId":"65846","inReplyTo":"6151b8e0a3989a51e6d9717e0ceac439f26f1c1d.1782303254.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 2/7] t6600: add test cases for side-exhaustion edge cases","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-24T13:43:53Z","receivedAt":"2026-06-24T13:43:56Z","isPatch":true,"body":"On 6/24/2026 8:14 AM, Elijah Newren via GitGitGadget wrote:\n> From: Elijah Newren \n> \n> Add test cases to t6600-test-reach.sh that exercise edge cases in the\n> side-exhaustion optimization for paint_down_to_common():\n> \n> - in_merge_bases_many:self: commit is both A and one of the X inputs\n> - get_merge_bases_many:duplicate-twos: duplicate entries in X list\n> - get_merge_bases_many:pending-stale: STALE transition on an\n> already-painted commit (ps-* diamond topology)\n> - get_merge_bases_many:infinity-both-sides: both tips outside the\n> commit-graph with non-monotonic dates (pi-* topology)\n\nI'm happy that these cases now exist.\n\n> +test_expect_success 'in_merge_bases_many:self' '\n> +\tcat >input <<-\\EOF &&\n> +\tA:commit-6-8\n> +\tX:commit-5-9\n> +\tX:commit-6-8\n> +\tEOF\n> +\techo \"in_merge_bases_many(A,X):1\" >expect &&\n> +\ttest_all_modes in_merge_bases_many\n> +'\n\nand using 'test_all_modes' is great to get coverage of all the\ndifferent commit-graph states. In reply to patch v2 4/7 I ask\nto see the results of the traces in these kinds of test cases,\nbut each of these modes will have different values.\n\nOne way to make these tests have potential to check exact stats\nwithout too much extra work would be to update 'test_all_modes'\nto run each command with GIT_TRACE2_EVENT set to a known trace\nfile (reset each time) that can then be checked after verifying\nthat the results of each command is the same.\n\nThen, these tests could have lines such as\n\n\ttest_trace2_data paint_down_to_common steps 20 Termination\n> -----------\n> \n> -The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n> -tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n> -so far. Once that commit is dequeued, every remaining entry is known\n> -to be STALE and the loop terminates. Specifically, the main loop\n> +The walk tracks the number of commits of each type in the queue\n> +(PARENT1-only, PARENT2-only, pending merge-base). The main loop\n> ends when one of the following conditions holds:\n> \n> 1. The queue is empty.\n> - 2. `max_nonstale` has been dequeued, meaning the queue only contains\n> - STALE entries.\n> + 2. The queue contains only stale entries.\n\nI'm grateful to see these changes happening to the doc in real-\ntime. I know it was extra work, but I'm grateful right now.\n\nHopefully future historians will also benefit from this effort.\n\n> +static void paint_count_update(struct paint_state *state,\n> +\t\t\t unsigned flags, int delta)\n> +{\n> +\tswitch (flags & (PARENT1 | PARENT2 | STALE)) {\n> +\tcase PARENT1:\n> +\t\tstate->p1_count += delta;\n> +\t\tbreak;\n> +\n> +\tcase PARENT2:\n> +\t\tstate->p2_count += delta;\n> +\t\tbreak;\n> +\n> +\tcase PARENT1 | PARENT2:\n> +\t\tstate->pending_merge_bases += delta;\n> +\t\tbreak;\n> +\n> +\tcase PARENT1 | PARENT2 | STALE:\n> +\t\tbreak;\n> +\n> +\tdefault:\n> +\t\tBUG(\"unexpected paint state\");\n> +\t}\n> +}\n\nI like the use of 'delta' to allow reuse of this switch.\n\n> +\n> +static void paint_queue_put(struct paint_state *state,\n> +\t\t\t struct commit *c, unsigned add_flags)\n> +{\n> +\tunsigned old_flags = c->object.flags;\n> +\tc->object.flags |= add_flags;\n> +\n> +\tif (old_flags & ENQUEUED) {\n> +\t\tpaint_count_update(state, old_flags, -1);\n> +\t\tpaint_count_update(state, c->object.flags, 1);\n> +\t} else {\n> +\t\tc->object.flags |= ENQUEUED;\n> +\t\tprio_queue_put(&state->queue, c);\n> +\t\tpaint_count_update(state, c->object.flags, 1);\n> +\t}\n> +}\n\nok: if we are already in the queue then we have old flags and\nmay need to subtract their values because they were counted\nalready. Otherwise, we need to queue it for the first time and\nonly add the values. Makes sense.\n\n> +\n> +static struct commit *paint_queue_get(struct paint_state *state)\n> +{\n\nSince we are going to make this a more complete termination\ncondition, we may want to make that very explicit with a doc-\ncomment. Something along the lines of \"dequeue a commit when\npossible, but also signal termination of the walk when we\nconclude that no more merge bases will be discovered due to\ninternal state.\"\n\n> @@ -187,12 +253,11 @@ static int paint_down_to_common(struct repository *r,\n> \t\t\t\treturn error(_(\"could not parse commit %s\"),\n> \t\t\t\t\t oid_to_hex(&p->object.oid));\n> \t\t\t}\n> -\t\t\tp->object.flags |= flags;\n> -\t\t\tnonstale_queue_put_dedup(&queue, p);\n> +\t\t\tpaint_queue_put(&state, p, flags);\n\nI like how this simplifies the flag-assignment logic somewhat.\n\nYou mentioned in your cover letter how the min_generation value\ncan add extra termination conditions. It may be a good idea to\ninsert min_generation into the paint_queue struct and make it a\ntermination condition for paint_queue_get(). If you consider this\ndirection, then I'd make it a separate patch on top of this one\n_before_ adding the one-sided change. The extra tests that cover\nthe exact number of walked commits can help to guarantee the same\nbehavior, assuming that some of those tests check a non-zero\nmin_generation input. (It may be good to add such trace tests in\nan earlier patch to help confidence in this case.)\n\nThanks,\n-Stolee\n"},{"id":"546324","messageId":"a029b1ff-07ac-4199-96b7-e85a76d957e3@gmail.com","threadId":"65846","inReplyTo":"8c72f01083237c00397dd074beda8f854e882cbe.1782303254.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 6/7] commit-reach: remove unused nonstale_queue dedup wrappers","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-24T13:55:29Z","receivedAt":"2026-06-24T13:55:34Z","isPatch":true,"body":"On 6/24/2026 8:14 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n> \n> nonstale_queue_put_dedup() and nonstale_queue_get_dedup() became\n> unused after the previous commit. The core nonstale_queue functions\n> remain in use by ahead_behind().\nThis is a nice cleanup that makes the previous diff easier to\nread. Thanks!\n"},{"id":"546325","messageId":"6b0d81e7-7617-4fb4-9e39-cdf8bc778837@gmail.com","threadId":"65846","inReplyTo":"d84b932e5b078edc8255b6944ecb67fc1aa086b0.1782303254.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 7/7] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-24T14:02:03Z","receivedAt":"2026-06-24T14:02:08Z","isPatch":true,"body":"On 6/24/2026 8:14 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n> \n> Add an early termination check to paint_down_to_common() using the\n> per-side counters introduced earlier. Once the walk enters the\n> finite-generation region, terminate early when one side's exclusive\n> count drops to zero -- no new merge-base can form without both paint\n> sides meeting.\n\nHaving this as the last patch is truly a nice climax moment for the\npatch series!\n\n> @@ -94,6 +94,9 @@ ends when one of the following conditions holds:\n> \n> 1. The queue is empty.\n> 2. The queue contains only stale entries.\n> + 3. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n> + remain in the queue, no pending merge-base candidates exist,\n> + and the walk has entered the finite-generation region.\n...> +Side-exhaustion condition\n> +~~~~~~~~~~~~~~~~~~~~~~~~~\n> +A new merge-base requires commits from both sides to meet. When one\n> +side's exclusive counter reaches zero and there are no pending\n> +merge-base candidates, no future traversal step can produce a new\n> +candidate.\n> +\n> +This optimization only activates in the finite-generation region\n> +where topological ordering holds. In that region, children are\n> +always visited before parents, so paint flags are final at visit\n> +time and an exhausted side cannot reappear. In the INFINITY region,\n> +commit-date ordering can violate this guarantee, so the check is\n> +skipped.\n> +\n\nAnd these doc updates inline make me happy.\n\n> Related documentation\n> ---------------------\n> \n> diff --git a/commit-reach.c b/commit-reach.c\n> index e0d9874f99..f79d0b64d6 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -133,17 +133,30 @@ static void paint_queue_put(struct paint_state *state,\n> \n> static struct commit *paint_queue_get(struct paint_state *state)\n> {\n> -\tstruct commit *commit;\n> +\tstruct commit *commit = prio_queue_get(&state->queue);\n> \n> -\tif (!state->p1_count && !state->p2_count &&\n> -\t !state->pending_merge_bases)\n> +\tif (!commit)\n> \t\treturn NULL;\nI see how the previous implementation has a termination condition\nbefore calling prio_queue_get(), which is technically more\nefficient. It does make this initial diff a bit more complicated\nbecause we are moving the prio_queue_get() line.\n\nIf the introduction of the method in patch 5/7 looked like this:\n\n+static struct commit *paint_queue_get(struct paint_state *state)\n+{\n+\tstruct commit *commit = prio_queue_get(&state->queue);\n+\n+\tif (!commit)\n+\t\treturn NULL;\n+\n+\tif (!state->p1_count && !state->p2_count &&\n+\t !state->pending_merge_bases)\n+\t\treturn NULL;\n+\n+\tcommit->object.flags &= ~ENQUEUED;\n+\tpaint_count_update(state, commit->object.flags, -1);\n+\treturn commit;\n+}\n\nThen this diff would look cleaner.\n\n(This is the nittiest of nitpicks so feel free to ignore if this\ndoesn't bother you at all.)\n\n> -\tcommit = prio_queue_get(&state->queue);\n> -\tif (commit) {\n> -\t\tcommit->object.flags &= ~ENQUEUED;\n> -\t\tpaint_count_update(state, commit->object.flags, -1);\n> +\tcommit->object.flags &= ~ENQUEUED;\n> +\n> +\tif (!state->pending_merge_bases) {\n> +\t\tif (!state->p1_count && !state->p2_count)\n> +\t\t\treturn NULL;\n> +\t\t/*\n> +\t\t * Side exhaustion: a new merge-base can only form\n> +\t\t * when both PARENT1-only and PARENT2-only commits\n> +\t\t * remain in the queue. In the finite-generation\n> +\t\t * region the queue is ordered topologically, so\n> +\t\t * no future step can add paint to visited commits\n> +\t\t * and an exhausted side cannot reappear.\n> +\t\t */\n> +\t\tif ((!state->p1_count || !state->p2_count) &&\n> +\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n> +\t\t\treturn NULL;\n> \t}\n> +\n> +\tpaint_count_update(state, commit->object.flags, -1);\n> \treturn commit;\n> }\n\nI like how the crux of this implementation is entirely within\npaint_queue_get() now.\n\n> diff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\n> index c1109fb42f..03175befb3 100755\n> --- a/t/t6600-test-reach.sh\n> +++ b/t/t6600-test-reach.sh\n> @@ -332,12 +332,12 @@ test_expect_success 'merge-base --all commit-walk steps' '\n> \tcp commit-graph-full .git/objects/info/commit-graph &&\n> \tGIT_TRACE2_EVENT=\"$(pwd)/trace-full.txt\" \\\n> \t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n> -\ttest_trace2_data paint_down_to_common steps 80 +\ttest_trace2_data paint_down_to_common steps 9 \n> \tcp commit-graph-half .git/objects/info/commit-graph &&\n> \tGIT_TRACE2_EVENT=\"$(pwd)/trace-half.txt\" \\\n> \t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n> -\ttest_trace2_data paint_down_to_common steps 81 +\ttest_trace2_data paint_down_to_common steps 57 '\nI love to see these steps change. If you take my suggestion to\nupdate more tests with these checks, then this diff will get bigger\n(but in a deserved way).\n\nAlso, when I suggested that 'test_all_modes' creates the trace\nfiles on our behalf, I forgot to mention that this specific test\nthat you added in patch 4/7 simplifies by running the merge-base\ncheck under 'test_all_modes' and then checking the trace2 data\non the three well-known files afterwards.\n\nThanks,\n-Stolee\n\n"},{"id":"546326","messageId":"c1368af3-2525-4911-8a42-2188a37f2f15@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 0/7] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-24T14:09:54Z","receivedAt":"2026-06-24T14:09:56Z","isPatch":true,"body":"On 6/24/2026 8:14 AM, Kristofer Karlsson via GitGitGadget wrote:\n> commit-reach: terminate merge-base walk when one side is exhausted\n> \n> Optimize paint_down_to_common() for merge-base queries that hit large\n> one-sided histories.\nI completed my review of this version. All of my comments are around\neither making the commit history a little cleaner or expanding the\ntests that use the trace2 data.\n\nI believe that this code is _correct_ and could be shipped as-is. My\ncomments are focused on making it the best that it could be, with an\neye towards a cleaner final result or a more robust test setup.\n\nThe most actionable things are:\n\n1. You can add tracing before the new tests, allowing the new tests\n to also check the step counts in their first versions and then\n get updated in the final patch to demonstrate how they change\n with that behavior change.\n\n2. The t6600 helper 'test_all_modes' could set GIT_TRACE2_EVENT for\n each mode into a different trace file that can be scanned later.\n This will simplify your current tracing tests but also unlock\n easier tracing like this in the future.\n\n3. The termination condition depending on min_generation could be\n refactored into paint_queue_get() to help make things even more\n obvious as to when we terminate. This should help with your\n concerns that you mentioned in response to patch 2/6 of the\n previous version:\n\n> I am not 100% happy with the halt-condition placement yet --\n> the existing loop in master already has several exit paths\n> (while condition, min_generation break, FIND_ALL break) and I\n> think there is an opportunity to consolidate them. But that is\n> a separate discussion and I do not want to derail this series.\n> I can propose some alternatives in a follow-up after this\n> lands.\n\nI then have some super minor comments around making the diffs\neven easier to read, but they could be ignored as they are very\nnit-picky. It's the kind of detail that I would try to resolve\nif I was the author, but I'm _not_. You are. Your time is\nvaluable so make your own conclusions as to whether you want to\ngo down that road. You've already entertained my ideas around\nupdating the docs as the implementation changes.\n\nThanks,\n-Stolee\n\n"},{"id":"546327","messageId":"CAL71e4MnA36ZchLaUsMSoLcb9LO77aac274jES8+oV=yxuigOA@mail.gmail.com","threadId":"65846","inReplyTo":"67c00a9f-2aa2-4e83-9c0a-317ca589b232@gmail.com","subject":"Re: [PATCH v2 0/7] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-24T14:25:46Z","receivedAt":"2026-06-24T14:25:58Z","isPatch":true,"body":"On Wed, 24 Jun 2026 at 15:34, Derrick Stolee wrote:\n>\n> I like seeing these updates including the deterministic steps. Is there\n> a reason you don't include the step data for 'merge-tree (across import)'\n> in your monorepo case? The wall-clock is substantial, so it's not like the\n> last two examples in git.git where there may not be any difference.\n\nI will have to attribute to laziness I suppose :)\nI ran the initial benchmarks before adding the trace, and I didn't\nupdate all of them,\njust enough to show the improvement and value of the trace data.\n\nI will ensure that I include all of it in the next version though\n(maybe 1-2 days from now?) or maybe drop some of the benchmarks to\nnot overload with partly redundant information.\n(merge-tree benchmarks doesn't perhaps add much significance on top\nof merge-base in practice).\n\nThanks,\nKristofer\n"},{"id":"546328","messageId":"CAL71e4O7s7y+SJRp3GZB+j9SLB_q=kK8ysCKqH9Mp2VDn0sT=Q@mail.gmail.com","threadId":"65846","inReplyTo":"560c91df-3c07-4c8f-9924-ef0cc7646e08@gmail.com","subject":"Re: [PATCH v2 4/7] commit-reach: add trace2 instrumentation to paint_down_to_common()","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-24T14:31:26Z","receivedAt":"2026-06-24T14:31:39Z","isPatch":true,"body":"On Wed, 24 Jun 2026 at 15:41, Derrick Stolee wrote:\n>\n> (highlighting this chunk)\n>\n> > + rm -rf .git/objects/info/commit-graph \\\n> > + .git/objects/info/commit-graphs &&\n> > +\n> > + GIT_TRACE2_EVENT=\"$(pwd)/trace-none.txt\" \\\n> > + git merge-base --all commit-9-9 commit-9-1 >actual &&\n> > + test_trace2_data paint_down_to_common steps 81 \n> I'd rather see the whitespace line before the `rm` to make it\n> more obvious that it's setting up the \"none\" case.\n\nAh yes, good point, will fix.\n\n> > + cp commit-graph-full .git/objects/info/commit-graph &&\n> > + GIT_TRACE2_EVENT=\"$(pwd)/trace-full.txt\" \\\n> > + git merge-base --all commit-9-9 commit-9-1 >actual &&\n> > + test_trace2_data paint_down_to_common steps 80 > +\n> > + cp commit-graph-half .git/objects/info/commit-graph &&\n> > + GIT_TRACE2_EVENT=\"$(pwd)/trace-half.txt\" \\\n> > + git merge-base --all commit-9-9 commit-9-1 >actual &&\n> > + test_trace2_data paint_down_to_common steps 81 > +'\n> > +\n>\n> This test is a great example. I look forward to seeing that it\n> updates in the future.\n>\n> One thing I was hoping to see was that your side-exhaustion tests\n> (from patch v2 2/7) would also include these checks so they are\n> more obviously updating when the implementation updates later.\n\nI was internally contemplating how much I should introduce the steps\nvalidation to existing tests. My worry was that it might make tests\nfragile - for example I repeatedly got some off-by-one changes\nafter refactoring the halt condition slightly (differs depending on\nadding the halts solely within paint_queue_get or having it at the end\nof the loop) and I think potentially other future work could affect it.\n\nBut I'm happy to attach the steps checks for more relevant tests,\nit's not much work to change.\n\n> One way to accomplish that is to reorder this patch before adding\n> those tests so their first version includes these checks and then\n> the values update when changing the implementation.\n\nI was thinking I could keep the same order, but the patch to introduce\nthe trace could also modify the tests at the same time - that would\nperhaps make it even more clear. Also this means I could avoid\nmaking changes to Elijah's commit that I already _partly_ butchered\n(extracted the test change as-is, but dropped the other file changes)\nand I don't want to make that one more unclean.\n\nThanks,\nKristofer\n"},{"id":"546329","messageId":"CAL71e4MCXUnRddRVGT2jcN2YT7bbfkOmFEdriQXNPk+9W_oLtg@mail.gmail.com","threadId":"65846","inReplyTo":"b4b33635-1279-46c0-819a-d29cc13921f5@gmail.com","subject":"Re: [PATCH v2 2/7] t6600: add test cases for side-exhaustion edge cases","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-24T14:33:51Z","receivedAt":"2026-06-24T14:34:06Z","isPatch":true,"body":"On Wed, 24 Jun 2026 at 15:43, Derrick Stolee wrote:\n>\n> One way to make these tests have potential to check exact stats\n> without too much extra work would be to update 'test_all_modes'\n> to run each command with GIT_TRACE2_EVENT set to a known trace\n> file (reset each time) that can then be checked after verifying\n> that the results of each command is the same.\n>\n> Then, these tests could have lines such as\n>\n> test_trace2_data paint_down_to_common steps 20 test_trace2_data paint_down_to_common steps 30 test_trace2_data paint_down_to_common steps 40 \n> after the test_all_modes line.\n\nThat does indeed look quite clean, I will try to massage the tests to\nutilize that, though I haven't fully worked it through in my head yet so\nI am not sure if/where I would get stuck. :)\n\nThanks,\nKristofer\n"},{"id":"546330","messageId":"CAL71e4N88H_VLd8nNfEVGqegbjT0bjQBgRdBN-kp1Y_U8ejJYg@mail.gmail.com","threadId":"65846","inReplyTo":"19639ad3-2d16-4f3b-be79-138e00144ea3@gmail.com","subject":"Re: [PATCH v2 5/7] commit-reach: introduce struct paint_state with per-side counters","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-24T14:38:50Z","receivedAt":"2026-06-24T14:39:02Z","isPatch":true,"body":"On Wed, 24 Jun 2026 at 15:54, Derrick Stolee wrote:\n>\n> I'm grateful to see these changes happening to the doc in real-\n> time. I know it was extra work, but I'm grateful right now.\n>\n> Hopefully future historians will also benefit from this effort.\n\nIt was honestly not bad at all, and I agree it felt quite nice to\nsee how the doc naturally changed along with the implementation.\n\n> > +static struct commit *paint_queue_get(struct paint_state *state)\n> > +{\n>\n> Since we are going to make this a more complete termination\n> condition, we may want to make that very explicit with a doc-\n> comment. Something along the lines of \"dequeue a commit when\n> possible, but also signal termination of the walk when we\n> conclude that no more merge bases will be discovered due to\n> internal state.\"\n\nYes, I'll make sure to clean that part up more, maybe also\nrename the function to be more descriptive.\n\n> You mentioned in your cover letter how the min_generation value\n> can add extra termination conditions. It may be a good idea to\n> insert min_generation into the paint_queue struct and make it a\n> termination condition for paint_queue_get(). If you consider this\n> direction, then I'd make it a separate patch on top of this one\n> _before_ adding the one-sided change. The extra tests that cover\n> the exact number of walked commits can help to guarantee the same\n> behavior, assuming that some of those tests check a non-zero\n> min_generation input. (It may be good to add such trace tests in\n> an earlier patch to help confidence in this case.)\n\nI think I might wait with this - the patch series already feels\nquite big, and I think it has a natural progression and finish now.\nBut I can definitely commit to following up later -- it would be a\nsmaller series that is easier to reason about, likely a single commit.\n\nThanks,\nKristofer\n"},{"id":"546331","messageId":"CAL71e4N1zMz=v9umGdGPTvLP1nF-tNLVQc+vAEBnekt2L0b6zQ@mail.gmail.com","threadId":"65846","inReplyTo":"6b0d81e7-7617-4fb4-9e39-cdf8bc778837@gmail.com","subject":"Re: [PATCH v2 7/7] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-24T14:47:13Z","receivedAt":"2026-06-24T14:47:25Z","isPatch":true,"body":"On Wed, 24 Jun 2026 at 16:02, Derrick Stolee wrote:\n>\n> I see how the previous implementation has a termination condition\n> before calling prio_queue_get(), which is technically more\n> efficient. It does make this initial diff a bit more complicated\n> because we are moving the prio_queue_get() line.\n\nI was thinking the efficiency here does not matter in practice -\nprio_queue_get() only returns NULL once, and all other times\nwhere we keep looping we do need the value.\n\nI agree it does get a bit complex though.\n\n> If the introduction of the method in patch 5/7 looked like this:\n>\n> +static struct commit *paint_queue_get(struct paint_state *state)\n> +{\n> + struct commit *commit = prio_queue_get(&state->queue);\n> +\n> + if (!commit)\n> + return NULL;\n> +\n> + if (!state->p1_count && !state->p2_count &&\n> + !state->pending_merge_bases)\n> + return NULL;\n> +\n> + commit->object.flags &= ~ENQUEUED;\n> + paint_count_update(state, commit->object.flags, -1);\n> + return commit;\n> +}\n>\n> Then this diff would look cleaner.\n>\n> (This is the nittiest of nitpicks so feel free to ignore if this\n> doesn't bother you at all.)\n\nThat's a good point. It doesn't technically bother me,\nbut it would be cleaner. The refactor commit would effectively\nbe looking into the future and prepare for it. I can change it for\nthe next version - my only thinking was that the current refactor\npatch matched my original idea for how to best handle\nthe halt condition, but that did indeed change after this discussion.\n\n> > - test_trace2_data paint_down_to_common steps 81 > + test_trace2_data paint_down_to_common steps 57 > '\n> I love to see these steps change. If you take my suggestion to\n> update more tests with these checks, then this diff will get bigger\n> (but in a deserved way).\n\nI will try to add them to some (but not all) tests since it's more\nclosely related to performance than correctness and I want to\navoid making too many tests overly fragile.\n\n> Also, when I suggested that 'test_all_modes' creates the trace\n> files on our behalf, I forgot to mention that this specific test\n> that you added in patch 4/7 simplifies by running the merge-base\n> check under 'test_all_modes' and then checking the trace2 data\n> on the three well-known files afterwards.\n\nThat's a nice bonus, I will try to see if I can manage to utilize it.\n\nThanks,\nKristofer\n"},{"id":"546332","messageId":"943b9360-68dd-404a-a129-6f857b3c16f5@gmail.com","threadId":"65846","inReplyTo":"CAL71e4N1zMz=v9umGdGPTvLP1nF-tNLVQc+vAEBnekt2L0b6zQ@mail.gmail.com","subject":"Re: [PATCH v2 7/7] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-24T15:07:52Z","receivedAt":"2026-06-24T15:07:54Z","isPatch":true,"body":"On 6/24/2026 10:47 AM, Kristofer Karlsson wrote:\n> On Wed, 24 Jun 2026 at 16:02, Derrick Stolee wrote:\n\n>>> - test_trace2_data paint_down_to_common steps 81 >> + test_trace2_data paint_down_to_common steps 57 >> '\n>> I love to see these steps change. If you take my suggestion to\n>> update more tests with these checks, then this diff will get bigger\n>> (but in a deserved way).\n> \n> I will try to add them to some (but not all) tests since it's more\n> closely related to performance than correctness and I want to\n> avoid making too many tests overly fragile.\nIn this case, I think it's more about protecting all of our special-\ncased termination conditions. The rigidity means that it is hard to\naccidentally change the behavior. It does have the downside that\nmore tests need to change if there is an intentional change, but it\nalso gives the same _evidence_ that the change has the intended\nimpact.\n\nWe are definitely leaning into personal preferences, though. There\nis no hard rule one way or another.\n\nThanks,\n-Stolee\n\n"},{"id":"546337","messageId":"xmqqbjczkgqm.fsf@gitster.g","threadId":"65846","inReplyTo":"19ed743bd10be5341eee040eb8070876b984773d.1782303254.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 1/7] Documentation/technical: add paint-down-to-common doc","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-06-24T17:09:21Z","receivedAt":"2026-06-24T17:09:23Z","isPatch":true,"body":"\"Kristofer Karlsson via GitGitGadget\" \nwrites:\n\n> From: Kristofer Karlsson \n>\n> Add a technical document describing the paint_down_to_common()\n> algorithm used for merge-base computation, covering the paint\n> walk, generation number regions, and termination conditions.\n>\n> Signed-off-by: Kristofer Karlsson \n> ---\n> Documentation/Makefile | 1 +\n> Documentation/technical/meson.build | 1 +\n> .../technical/paint-down-to-common.adoc | 114 ++++++++++++++++++\n> commit-reach.c | 6 +-\n> 4 files changed, 121 insertions(+), 1 deletion(-)\n> create mode 100644 Documentation/technical/paint-down-to-common.adoc\n\nGreat write-up that very clearly and concisely explains what goes on\ninside the merge-base computation. Thanks for a pleasant read.\n"},{"id":"546459","messageId":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v2.git.1782303254.gitgitgadget@gmail.com","subject":"[PATCH v3 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:07:57Z","receivedAt":"2026-06-26T13:08:09Z","isPatch":true,"body":"commit-reach: terminate merge-base walk when one paint side is exhausted\n\nOptimize paint_down_to_common() for merge-base queries that hit large\none-sided histories.\n\nWhen the walk from one side reaches a commit with a very low generation\nnumber that the other side never paints, the walk is forced to drain most of\nthe graph. A common trigger is a repository import that grafts a separate\nhistory with its own root, but any merge that introduces a low-generation\ncommit never painted by the other side has the same effect.\n\nA new merge-base candidate can only be discovered when exclusive PARENT1 and\nPARENT2 paint meet. This series teaches paint_down_to_common() to stop as\nsoon as one side has no exclusive commits left in the queue; once one side\nis exhausted, no further candidates can appear.\n\n origin/HEAD o o PR HEAD\n | |\n (import) o :\n / \\ /\n | o merge-base\n | |\n : : (~2.5M commits)\n | |\n import root main root\n\n\nIn the RFC thread [1], Derrick Stolee provided a criss-cross counterexample\nthat sharpened the halt condition, and Elijah Newren independently\ndiscovered the same optimization and shared an implementation in PR #2150\n[2]. Patches 2-3 incorporate test cases from Elijah's branch.\n\nThis series implements the optimization only after the walk enters the\nfinite-generation region, where generation ordering guarantees that paint on\nvisited commits is final.\n\nPatch layout:\n\n1/8 Documentation/technical: add paint-down-to-common doc 2/8 t6600: add\ntest cases for side-exhaustion edge cases 3/8 t6099, t6600: add\nside-exhaustion regression tests 4/8 commit-reach: add trace2\ninstrumentation to paint_down_to_common() 5/8 commit-reach: introduce struct\npaint_state with per-side counters 6/8 commit-reach: remove unused\nnonstale_queue dedup wrappers 7/8 commit-reach: terminate merge-base walk\nwhen one paint side is exhausted 8/8 commit-reach: move min_generation check\ninto paint_queue_get()\n\nBenchmarks\n\nStep counts are deterministic (measured via trace2_data_intmax added in\npatch 4). Wall-clock times are best-of-11 runs.\n\n2.6M-commit monorepo with commit-graph:\n\n steps wall-clock\n merge-base --all (across import) 2143438 -> 3 3.67s -> 5ms\n merge-base --all (1000 apart) 2692915 -> 1035 4.41s -> 7ms\n merge-base --all (5000 apart) 2692915 -> 6401 4.45s -> 13ms\n merge-base --all (HEAD vs import) 2698872 -> 45960 4.50s -> 79ms\n merge-tree (across import) 2143438 -> 3 4.42s -> 11ms\n\n\ngit.git (88k commits, commit-graph):\n\n steps wall-clock\n merge-base --all v2.0.0 v2.55.0-rc1 72264 -> 44589 110ms -> 68ms\n merge-base --all HEAD HEAD~1000 9891 -> 3828 18ms -> 10ms\n merge-base --all HEAD HEAD~10000 72303 -> 41487 101ms -> 50ms\n\n\nChanges since v2:\n\n * New patch 8/8: moved the min_generation termination check and the\n last_gen monotonicity assertion into paint_queue_get(), consolidating\n halt conditions. commit_graph_generation() is now called once per\n dequeued commit and shared across all checks.\n\n * Widened the generation-monotonicity BUG assertion to fire\n unconditionally, not only when min_generation is set. The side-exhaustion\n optimization depends on correct generation ordering, so the assertion\n should always be active. This is a behavior change: the BUG() now fires\n for any generation ordering violation, regardless of the caller.\n\n * Moved all halt conditions inside paint_queue_get() with the \"pop first\"\n form: pop, check, then decrement counters. This keeps the optimization\n commit's diff minimal (just inserting the new checks between pop and\n decrement).\n\n * Shortened the doc comment on paint_queue_get() to describe what it does\n rather than how. Inline comments on each return NULL explain the specific\n halt condition.\n\n * Replaced the manual commit-graph setup in the step-count test with\n run_all_modes, which now sets GIT_TRACE2_EVENT per mode and produces\n trace-mode-{none,full,half,no-gdat}.txt files.\n\n * Added a test_paint_down_steps helper for concise 4-mode step assertions\n with diagnostic output on mismatch (prints \"expected X, got Y\" instead of\n a silent grep failure).\n\n * Added step-count assertions to the single-walk edge-case tests:\n in_merge_bases_many:self, pending-stale, infinity-both-sides,\n mixed-finite-infinity.\n\n * Included step counts alongside wall-clock times in the benchmark tables.\n\nChanges since v1:\n\n * Reordered patches: documentation first (describing the existing\n algorithm), tests before code changes, so they demonstrate passing with\n old logic first.\n\n * Dropped the ahead_behind decoupling patch. paint_state is now a NEW\n struct alongside nonstale_queue instead of replacing it. ahead_behind()\n is completely untouched.\n\n * Removed nonstale_queue_put_dedup() and nonstale_queue_get_dedup() (dead\n code after the conversion) in a separate commit.\n\n * Renamed: struct paint_queue -> paint_state, field pq -> queue,\n paint_count_add/remove -> paint_count_update (single function with signed\n delta parameter).\n\n * Split the old paint_count_transition (which handled both old and new\n flags in one call) into separate remove/add calls with a signed delta.\n This eliminates the need for the case 0 handler (which tracked \"not in\n the queue\") and allows an exhaustive switch on (PARENT1 | PARENT2 |\n STALE) that documents all valid flag combinations, with BUG() in default.\n\n * Added trace2_data_intmax() instrumentation to report the number of\n commits visited per paint walk (separate commit), with deterministic\n step-count assertions in t6600.\n\n * Expanded switch statements to multi-line format per .clang-format.\n\n * Used !count style throughout instead of count == 0.\n\n * Updated technical documentation alongside code changes.\n\n[1]\nhttps://lore.kernel.org/git/CAL71e4Ps-2_0+uuZu43N9pFnXBemoAohPs_eyRJf8taXHJPAXQ@mail.gmail.com/T/#u\n[2] https://github.com/gitgitgadget/git/pull/2150\n\nElijah Newren (1):\n t6600: add test cases for side-exhaustion edge cases\n\nKristofer Karlsson (7):\n Documentation/technical: add paint-down-to-common doc\n t6099, t6600: add side-exhaustion regression tests\n commit-reach: add trace2 instrumentation to paint_down_to_common()\n commit-reach: introduce struct paint_state with per-side counters\n commit-reach: remove unused nonstale_queue dedup wrappers\n commit-reach: terminate merge-base walk when one paint side is\n exhausted\n commit-reach: move min_generation check into paint_queue_get()\n\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 137 +++++++++++++\n commit-reach.c | 147 ++++++++++----\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 ++++++++\n t/t6600-test-reach.sh | 181 ++++++++++++++++--\n 7 files changed, 501 insertions(+), 49 deletions(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\n\nbase-commit: 6c3d7b73556db708feb3b16232fab1efc4353428\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2149%2Fspkrka%2Fside-exhaust-pr-v3\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2149/spkrka/side-exhaust-pr-v3\nPull-Request: https://github.com/gitgitgadget/git/pull/2149\n\nRange-diff vs v2:\n\n 1: 19ed743bd1 = 1: 2593866bce Documentation/technical: add paint-down-to-common doc\n 2: 6151b8e0a3 = 2: 9efc084850 t6600: add test cases for side-exhaustion edge cases\n 3: 90f09ecb5c = 3: 14b0d86b93 t6099, t6600: add side-exhaustion regression tests\n 4: 6ade4df2ed ! 4: 2592264cda commit-reach: add trace2 instrumentation to paint_down_to_common()\n @@ Commit message\n \n Add a step counter and trace2_data_intmax() call so that the number\n of commits visited during the paint walk is observable via\n - GIT_TRACE2_PERF. This provides a way to measure the impact of\n + GIT_TRACE2_EVENT. This provides a way to measure the impact of\n future optimizations without relying on wall-clock benchmarks alone.\n \n Signed-off-by: Kristofer Karlsson \n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n }\n \n ## t/t6600-test-reach.sh ##\n -@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n - \ttest_all_modes get_merge_bases_many\n +@@ t/t6600-test-reach.sh: test_expect_success 'setup' '\n '\n \n -+test_expect_success 'merge-base --all commit-walk steps' '\n -+\ttest_when_finished rm -rf .git/objects/info/commit-graph \\\n -+\t\t.git/objects/info/commit-graphs &&\n -+\trm -rf .git/objects/info/commit-graph \\\n -+\t\t.git/objects/info/commit-graphs &&\n -+\n -+\tGIT_TRACE2_EVENT=\"$(pwd)/trace-none.txt\" \\\n -+\t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n -+\ttest_trace2_data paint_down_to_common steps 81 actual &&\n +-\ttest_cmp expect actual &&\n +-\tcp commit-graph-full .git/objects/info/commit-graph &&\n +-\t\"$@\" actual &&\n +-\ttest_cmp expect actual &&\n +-\tcp commit-graph-half .git/objects/info/commit-graph &&\n +-\t\"$@\" actual &&\n +-\ttest_cmp expect actual &&\n +-\tcp commit-graph-no-gdat .git/objects/info/commit-graph &&\n +-\t\"$@\" actual &&\n +-\ttest_cmp expect actual\n ++\tgraph=.git/objects/info/commit-graph &&\n ++\ttest_when_finished rm -rf \"$graph\" \"${graph}s\" &&\n ++\trm -f trace-mode-*.txt &&\n +\n -+\tcp commit-graph-full .git/objects/info/commit-graph &&\n -+\tGIT_TRACE2_EVENT=\"$(pwd)/trace-full.txt\" \\\n -+\t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n -+\ttest_trace2_data paint_down_to_common steps 80 /dev/null ||\n ++\t\ttrue &&\n ++\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace-mode-${mode}.txt\" \\\n ++\t\t\t\"$@\" actual &&\n ++\t\ttest_cmp expect actual || return 1\n ++\tdone\n + }\n + \n + test_all_modes () {\n + \trun_all_modes test-tool reach \"$@\"\n + }\n + \n ++test_paint_down_steps () {\n ++\tfor mode in none full half no-gdat\n ++\tdo\n ++\t\ttest_trace2_data paint_down_to_common steps \"$1\" \\\n ++\t\t\t<\"trace-mode-${mode}.txt\" || return 1\n ++\t\tshift\n ++\tdone\n ++}\n +\n -+\tcp commit-graph-half .git/objects/info/commit-graph &&\n -+\tGIT_TRACE2_EVENT=\"$(pwd)/trace-half.txt\" \\\n -+\t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n -+\ttest_trace2_data paint_down_to_common steps 81 input <<-\\EOF &&\n + \tA:commit-5-7\n +@@ t/t6600-test-reach.sh: test_expect_success 'in_merge_bases_many:self' '\n + \tX:commit-6-8\n + \tEOF\n + \techo \"in_merge_bases_many(A,X):1\" >expect &&\n +-\ttest_all_modes in_merge_bases_many\n ++\ttest_all_modes in_merge_bases_many &&\n ++\ttest_paint_down_steps 45 2 25 3\n + '\n + \n + test_expect_success 'is_descendant_of:hit' '\n +@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:pending-stale' '\n + \t\techo \"get_merge_bases_many(A,X):\" &&\n + \t\tgit rev-parse ps-B\n + \t} >expect &&\n +-\ttest_all_modes get_merge_bases_many\n ++\ttest_all_modes get_merge_bases_many &&\n ++\ttest_paint_down_steps 6 6 6 6\n + '\n + \n + test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n +@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n + \t\techo \"get_merge_bases_many(A,X):\" &&\n + \t\tgit rev-parse pi-B\n + \t} >expect &&\n +-\ttest_all_modes get_merge_bases_many\n ++\ttest_all_modes get_merge_bases_many &&\n ++\ttest_paint_down_steps 5 5 5 5\n + '\n + \n + test_expect_success 'setup mixed finite/INFINITY topology' '\n +@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n + \t\techo \"get_merge_bases_many(A,X):\" &&\n + \t\tgit rev-parse ps-X\n + \t} >expect &&\n +-\ttest_all_modes get_merge_bases_many\n ++\ttest_all_modes get_merge_bases_many &&\n ++\ttest_paint_down_steps 3 3 3 3\n +'\n +\n ++test_expect_success 'merge-base --all commit-walk steps' '\n ++\t>input &&\n ++\tgit rev-parse commit-9-1 >expect &&\n ++\trun_all_modes git merge-base --all commit-9-9 commit-9-1 &&\n ++\ttest_paint_down_steps 81 80 81 81\n + '\n + \n test_expect_success 'reduce_heads' '\n - \tcat >input <<-\\EOF &&\n - \tX:commit-1-10\n 5: f24edd45f0 ! 5: e82e0c72b6 commit-reach: introduce struct paint_state with per-side counters\n @@ Commit message\n (max_nonstale). This is equivalent behavior -- both conditions\n detect that no non-stale entries remain.\n \n - The existing nonstale_queue is left in place for ahead_behind().\n + paint_queue_get() uses a \"pop first\" form: it dequeues a commit,\n + then checks the counters. This means the loop exits one iteration\n + earlier than the old code in some topologies (the popped stale\n + commit is never processed), so a few step counts drop by one.\n \n - Step counts (via trace2 from the previous commit) are identical\n - before and after this refactoring, confirming no behavioral change.\n + The existing nonstale_queue is left in place for ahead_behind().\n \n Signed-off-by: Kristofer Karlsson \n \n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_q\n +\n +static struct commit *paint_queue_get(struct paint_state *state)\n +{\n -+\tstruct commit *commit;\n ++\tstruct commit *commit = prio_queue_get(&state->queue);\n ++\n ++\tif (!commit)\n ++\t\treturn NULL;\n ++\n ++\tcommit->object.flags &= ~ENQUEUED;\n +\n +\tif (!state->p1_count && !state->p2_count &&\n +\t !state->pending_merge_bases)\n +\t\treturn NULL;\n +\n -+\tcommit = prio_queue_get(&state->queue);\n -+\tif (commit) {\n -+\t\tcommit->object.flags &= ~ENQUEUED;\n -+\t\tpaint_count_update(state, commit->object.flags, -1);\n -+\t}\n ++\tpaint_count_update(state, commit->object.flags, -1);\n +\treturn commit;\n +}\n +\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n \ttrace2_data_intmax(\"paint_down_to_common\", r,\n \t\t\t \"steps\", steps);\n \tcommit_list_sort_by_date(result);\n +\n + ## t/t6600-test-reach.sh ##\n +@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:pending-stale' '\n + \t\tgit rev-parse ps-B\n + \t} >expect &&\n + \ttest_all_modes get_merge_bases_many &&\n +-\ttest_paint_down_steps 6 6 6 6\n ++\ttest_paint_down_steps 5 5 5 5\n + '\n + \n + test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n +@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n + \t\tgit rev-parse pi-B\n + \t} >expect &&\n + \ttest_all_modes get_merge_bases_many &&\n +-\ttest_paint_down_steps 5 5 5 5\n ++\ttest_paint_down_steps 5 4 5 5\n + '\n + \n + test_expect_success 'setup mixed finite/INFINITY topology' '\n 6: 8c72f01083 = 6: e6181bf3c1 commit-reach: remove unused nonstale_queue dedup wrappers\n 7: d84b932e5b ! 7: f3572a8a89 commit-reach: terminate merge-base walk when one paint side is exhausted\n @@ Commit message\n once the walk enters the finite-generation region where ordering\n guarantees hold.\n \n + Widen the existing generation-monotonicity BUG assertion to fire\n + unconditionally, not only when min_generation is set. The\n + side-exhaustion optimization depends on correct generation ordering,\n + so the assertion should always be active.\n +\n Step counts measured with trace2 on git.git with commit-graph:\n \n merge-base --all v2.0.0 v2.55.0-rc1:\n @@ Documentation/technical/paint-down-to-common.adoc: existing candidates by provin\n \n ## commit-reach.c ##\n @@ commit-reach.c: static void paint_queue_put(struct paint_state *state,\n + \t}\n + }\n \n ++/*\n ++ * Dequeue the next commit for the paint walk, or return NULL when\n ++ * no more merge bases can be discovered.\n ++ */\n static struct commit *paint_queue_get(struct paint_state *state)\n {\n --\tstruct commit *commit;\n -+\tstruct commit *commit = prio_queue_get(&state->queue);\n + \tstruct commit *commit = prio_queue_get(&state->queue);\n +@@ commit-reach.c: static struct commit *paint_queue_get(struct paint_state *state)\n + \n + \tcommit->object.flags &= ~ENQUEUED;\n \n -\tif (!state->p1_count && !state->p2_count &&\n -\t !state->pending_merge_bases)\n -+\tif (!commit)\n - \t\treturn NULL;\n - \n --\tcommit = prio_queue_get(&state->queue);\n --\tif (commit) {\n --\t\tcommit->object.flags &= ~ENQUEUED;\n --\t\tpaint_count_update(state, commit->object.flags, -1);\n -+\tcommit->object.flags &= ~ENQUEUED;\n -+\n +-\t\treturn NULL;\n +\tif (!state->pending_merge_bases) {\n ++\t\t/* only stale entries remain */\n +\t\tif (!state->p1_count && !state->p2_count)\n +\t\t\treturn NULL;\n -+\t\t/*\n -+\t\t * Side exhaustion: a new merge-base can only form\n -+\t\t * when both PARENT1-only and PARENT2-only commits\n -+\t\t * remain in the queue. In the finite-generation\n -+\t\t * region the queue is ordered topologically, so\n -+\t\t * no future step can add paint to visited commits\n -+\t\t * and an exhausted side cannot reappear.\n -+\t\t */\n ++\n ++\t\t/* one side is exhausted */\n +\t\tif ((!state->p1_count || !state->p2_count) &&\n +\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n +\t\t\treturn NULL;\n - \t}\n -+\n -+\tpaint_count_update(state, commit->object.flags, -1);\n ++\t}\n + \n + \tpaint_count_update(state, commit->object.flags, -1);\n \treturn commit;\n - }\n +@@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n + \t\ttimestamp_t generation = commit_graph_generation(commit);\n + \t\tsteps++;\n \n +-\t\tif (min_generation && generation > last_gen)\n ++\t\tif (generation > last_gen)\n + \t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n + \t\t\t generation, last_gen,\n + \t\t\t oid_to_hex(&commit->object.oid));\n \n ## t/t6600-test-reach.sh ##\n -@@ t/t6600-test-reach.sh: test_expect_success 'merge-base --all commit-walk steps' '\n - \tcp commit-graph-full .git/objects/info/commit-graph &&\n - \tGIT_TRACE2_EVENT=\"$(pwd)/trace-full.txt\" \\\n - \t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n --\ttest_trace2_data paint_down_to_common steps 80 expect &&\n + \ttest_all_modes in_merge_bases_many &&\n +-\ttest_paint_down_steps 45 2 25 3\n ++\ttest_paint_down_steps 45 1 25 1\n + '\n \n - \tcp commit-graph-half .git/objects/info/commit-graph &&\n - \tGIT_TRACE2_EVENT=\"$(pwd)/trace-half.txt\" \\\n - \t\tgit merge-base --all commit-9-9 commit-9-1 >actual &&\n --\ttest_trace2_data paint_down_to_common steps 81 input &&\n + \tgit rev-parse commit-9-1 >expect &&\n + \trun_all_modes git merge-base --all commit-9-9 commit-9-1 &&\n +-\ttest_paint_down_steps 81 80 81 81\n ++\ttest_paint_down_steps 81 9 57 10\n '\n \n test_expect_success 'reduce_heads' '\n -: ---------- > 8: 4b9f192d98 commit-reach: move min_generation check into paint_queue_get()\n\n-- \ngitgitgadget\n"},{"id":"546460","messageId":"2593866bce5d359f204c68925a94e14e9a807e1e.1782479286.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v3 1/8] Documentation/technical: add paint-down-to-common doc","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:07:58Z","receivedAt":"2026-06-26T13:08:09Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a technical document describing the paint_down_to_common()\nalgorithm used for merge-base computation, covering the paint\nwalk, generation number regions, and termination conditions.\n\nSigned-off-by: Kristofer Karlsson \n---\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 114 ++++++++++++++++++\n commit-reach.c | 6 +-\n 4 files changed, 121 insertions(+), 1 deletion(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 2699f0b24a..f8dea4b395 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -129,6 +129,7 @@ TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\n TECH_DOCS += technical/packfile-uri\n TECH_DOCS += technical/pack-heuristics\n+TECH_DOCS += technical/paint-down-to-common\n TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/platform-support\ndiff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\nindex ec07088c57..9ce11d5e48 100644\n--- a/Documentation/technical/meson.build\n+++ b/Documentation/technical/meson.build\n@@ -18,6 +18,7 @@ articles = [\n 'multi-pack-index.adoc',\n 'packfile-uri.adoc',\n 'pack-heuristics.adoc',\n+ 'paint-down-to-common.adoc',\n 'parallel-checkout.adoc',\n 'partial-clone.adoc',\n 'platform-support.adoc',\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nnew file mode 100644\nindex 0000000000..c10d5d2887\n--- /dev/null\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -0,0 +1,114 @@\n+Merge-Base Computation and paint_down_to_common()\n+==================================================\n+\n+The function `paint_down_to_common()` in `commit-reach.c` computes merge\n+bases by walking the commit graph backwards from two sets of tips and\n+finding where their ancestry meets.\n+\n+Use cases\n+---------\n+\n+Computing merge bases is used in two different ways:\n+\n+ 1. *Finding all merge bases* (`merge-base --all`, `merge-tree`,\n+ `merge`, `rebase`). A merge base is a common ancestor that is\n+ not itself an ancestor of another common ancestor.\n+\n+ 2. *Ancestry checks* (`in_merge_bases`, used by `merge-base\n+ --is-ancestor`, `branch -d`, `fetch`). These ask: \"is commit A\n+ an ancestor of commit B?\" If a common ancestor equals one of the\n+ inputs, that input is necessarily the only merge base -- no other\n+ common ancestor can be both as recent and not an ancestor of it.\n+\n+Both use cases share the same algorithm and implementation.\n+\n+Algorithm\n+---------\n+\n+Given a commit `one` and a set of commits `twos[]`, the walk paints\n+commits with two colors:\n+\n+ - PARENT1: reachable from `one`\n+ - PARENT2: reachable from any commit in `twos[]`\n+\n+The walk uses a priority queue ordered by generation number (falling\n+back to commit date when generation numbers are unavailable). Each\n+step dequeues the highest-priority commit (this is when we say a\n+commit is \"visited\") and propagates its paint flags to its parents,\n+enqueuing them if they gained new flags. When a commit receives\n+both PARENT1 and PARENT2, it is a merge-base candidate. A candidate\n+gains the STALE flag so its ancestors propagate staleness -- any\n+deeper common ancestor is necessarily redundant.\n+\n+INFINITY and finite generation regions\n+--------------------------------------\n+\n+The commit-graph stores a generation number for each commit. Commits\n+not in the commit-graph have generation `GENERATION_NUMBER_INFINITY`. The\n+graph is closed under reachability: if a commit is in the graph, all\n+its ancestors are too. This partitions the commit graph into two regions:\n+\n+....\n+ +---------------------------------------+\n+ | INFINITY region |\n+ | generation = INFINITY |\n+ | queue order: heuristic (commit date) |\n+ +---------------------------------------+\n+ |\n+ v\n+ +---------------------------------------+\n+ | Finite region |\n+ | generation = finite |\n+ | queue order: topological |\n+ +---------------------------------------+\n+....\n+\n+When the commit-graph is enabled, the INFINITY region is typically\n+very small -- it only contains commits added since the last\n+commit-graph refresh.\n+\n+All reachable INFINITY-generation commits are visited before any\n+finite-generation commit, because INFINITY is larger than any finite\n+value. Once the walk crosses into the finite region, it stays there.\n+\n+In the finite region, generation ordering guarantees topological\n+traversal: children are always visited before their parents. This\n+means that paint on already-visited commits is final -- no future\n+traversal step can add paint to them.\n+\n+In the INFINITY region, commit-date ordering can violate this: a\n+parent with a later date can be visited before a child with an earlier\n+date. Paint flags are therefore NOT final at visit time, and a\n+commit visited with only one side's paint may later gain the other.\n+\n+Paint flags are only added, never removed. Since each flag can be set\n+at most once per commit, the number of times a commit can be\n+re-enqueued is bounded by the number of flag transitions.\n+\n+Termination\n+-----------\n+\n+The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n+tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n+so far. Once that commit is dequeued, every remaining entry is known\n+to be STALE and the loop terminates. Specifically, the main loop\n+ends when one of the following conditions holds:\n+\n+ 1. The queue is empty.\n+ 2. `max_nonstale` has been dequeued, meaning the queue only contains\n+ STALE entries.\n+\n+Stale entry condition\n+~~~~~~~~~~~~~~~~~~~~~\n+Once all queued entries are stale, no new merge-base candidates can\n+be discovered -- that requires at least one non-stale commit from\n+each side meeting. Continuing the walk could still invalidate\n+existing candidates by proving one is an ancestor of another, but\n+`remove_redundant()` handles that as a post-processing step, so it\n+is safe to exit early.\n+\n+Related documentation\n+---------------------\n+\n+ - `Documentation/technical/commit-graph.adoc` -- generation numbers\n+ and the reachability closure property.\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 5df471a313..a9483759e0 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -96,7 +96,11 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n \treturn commit;\n }\n \n-/* all input commits in one and twos[] must have been parsed! */\n+/*\n+ * See Documentation/technical/paint-down-to-common.adoc\n+ *\n+ * All input commits in one and twos[] must have been parsed!\n+ */\n static int paint_down_to_common(struct repository *r,\n \t\t\t\tstruct commit *one, int n,\n \t\t\t\tstruct commit **twos,\n-- \ngitgitgadget\n\n"},{"id":"546461","messageId":"9efc08485081a60614ecf99b5a37ce0a54d76e82.1782479286.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v3 2/8] t6600: add test cases for side-exhaustion edge cases","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:07:59Z","receivedAt":"2026-06-26T13:08:10Z","isPatch":true,"body":"From: Elijah Newren \n\nAdd test cases to t6600-test-reach.sh that exercise edge cases in the\nside-exhaustion optimization for paint_down_to_common():\n\n - in_merge_bases_many:self: commit is both A and one of the X inputs\n - get_merge_bases_many:duplicate-twos: duplicate entries in X list\n - get_merge_bases_many:pending-stale: STALE transition on an\n already-painted commit (ps-* diamond topology)\n - get_merge_bases_many:infinity-both-sides: both tips outside the\n commit-graph with non-monotonic dates (pi-* topology)\n\nSigned-off-by: Elijah Newren \nSigned-off-by: Kristofer Karlsson \n---\n t/t6600-test-reach.sh | 111 ++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 111 insertions(+)\n\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex b5b314e570..c2e091aad1 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -49,6 +49,62 @@ test_expect_success 'setup' '\n \t\t\tgit tag -a -m \"$x-$i\" tag-$x-$i commit-$x-$i || return 1\n \t\tdone\n \tdone &&\n+\n+\t# Build a small side topology to exercise the (PARENT1|PARENT2) ->\n+\t# (PARENT1|PARENT2|STALE) transition in paint_down_to_common(); the\n+\t# 10x10 grid above does not exercise it because no merge-base candidate\n+\t# there is a descendant of another, so STALE never reaches a\n+\t# still-pending candidate.\n+\t#\n+\t# ps-X\n+\t# /|\\\n+\t# / | \\\n+\t# ps-Z ps-B ps-W\n+\t# | / \\ |\n+\t# | / \\ |\n+\t# |/ \\|\n+\t# ps-T1 ps-T2\n+\t#\n+\t# where ps-T1=merge(ps-Z,ps-B), ps-T2=merge(ps-W,ps-B), so\n+\t# merge-base(ps-T1,ps-T2) = ps-B. During the walk, ps-X transitions\n+\t# to (PARENT1|PARENT2) via ps-Z and ps-W before ps-B is dequeued;\n+\t# then the STALE-walk from ps-B transitions ps-X to\n+\t# (PARENT1|PARENT2|STALE).\n+\tgit checkout --orphan ps-orphan &&\n+\ttest_commit ps-X &&\n+\tgit checkout -b ps-B-br ps-X && test_commit ps-B &&\n+\tgit checkout -b ps-Z-br ps-X && test_commit ps-Z &&\n+\tgit checkout -b ps-W-br ps-X && test_commit ps-W &&\n+\tgit checkout -b ps-T1 ps-Z &&\n+\tgit merge --no-ff -m ps-T1 ps-B &&\n+\tgit checkout -b ps-T2 ps-W &&\n+\tgit merge --no-ff -m ps-T2 ps-B &&\n+\n+\t# Build a side topology that lives entirely outside the half\n+\t# commit-graph and has non-monotonic commit dates, to exercise the\n+\t# INFINITY-gate in paint_down_to_common. With both tips outside\n+\t# the graph, generation is INFINITY and the queue falls back to\n+\t# commit-date order, which here is non-monotonic.\n+\t#\n+\t# pi-X (date 500, PARENT1 tip) --> pi-P, pi-D\n+\t# pi-D (date 480) --> pi-C\n+\t# pi-C (date 200) --> pi-B\n+\t# pi-B (date 100, PARENT2 tip) --> pi-P\n+\t# pi-P (date 450, root)\n+\t#\n+\t# merge-base(pi-X, pi-B) = pi-B (it is an ancestor of pi-X and is\n+\t# itself one of the queried tips).\n+\tgit checkout --orphan pi-orphan &&\n+\ttest_commit --date \"@450 +0000\" pi-P &&\n+\ttest_commit --date \"@100 +0000\" pi-B &&\n+\ttest_commit --date \"@200 +0000\" pi-C &&\n+\ttest_commit --date \"@480 +0000\" pi-D &&\n+\tGIT_AUTHOR_DATE=\"@500 +0000\" GIT_COMMITTER_DATE=\"@500 +0000\" \\\n+\t\tgit commit-tree -p pi-D -p pi-P -m pi-X pi-D^{tree} >pi-X-oid &&\n+\tpi_x=\"$(cat pi-X-oid)\" &&\n+\tgit branch -f pi-X-br \"$pi_x\" &&\n+\tgit tag pi-X \"$pi_x\" &&\n+\n \tgit commit-graph write --reachable &&\n \tmv .git/objects/info/commit-graph commit-graph-full &&\n \tchmod u+w commit-graph-full &&\n@@ -146,6 +202,16 @@ test_expect_success 'in_merge_bases_many:miss-heuristic' '\n \ttest_all_modes in_merge_bases_many\n '\n \n+test_expect_success 'in_merge_bases_many:self' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-8\n+\tX:commit-5-9\n+\tX:commit-6-8\n+\tEOF\n+\techo \"in_merge_bases_many(A,X):1\" >expect &&\n+\ttest_all_modes in_merge_bases_many\n+'\n+\n test_expect_success 'is_descendant_of:hit' '\n \tcat >input <<-\\EOF &&\n \tA:commit-5-7\n@@ -183,6 +249,51 @@ test_expect_success 'get_merge_bases_many' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'get_merge_bases_many:duplicate-twos' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tX:commit-4-8\n+\tX:commit-4-8\n+\tX:commit-6-6\n+\tX:commit-6-6\n+\tX:commit-8-3\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse commit-5-6 \\\n+\t\t\t commit-4-7 | sort\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n+test_expect_success 'get_merge_bases_many:pending-stale' '\n+\t# Exercises the (PARENT1|PARENT2) -> (...|STALE) transition path in\n+\t# paint_down_to_common(). See the topology comment in the setup test.\n+\tcat >input <<-\\EOF &&\n+\tA:ps-T1\n+\tX:ps-T2\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse ps-B\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n+test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n+\t# Exercises the push-time INFINITY-gate in paint_down_to_common(). See\n+\t# the pi-* topology comment in the setup test.\n+\tcat >input <<-\\EOF &&\n+\tA:pi-X\n+\tX:pi-B\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse pi-B\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546462","messageId":"14b0d86b93c1c9e733e16b1d4786b227f3651c55.1782479286.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v3 3/8] t6099, t6600: add side-exhaustion regression tests","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:08:00Z","receivedAt":"2026-06-26T13:08:11Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd t6099 to test the case where multiple merge-base candidates exist\nand one is an ancestor of another. This exercises the side-exhaustion\noptimization in paint_down_to_common together with the\nremove_redundant safety net in get_merge_bases_many_0.\n\nAdd a mixed finite/INFINITY test to t6600 where one tip is outside\nthe commit-graph (INFINITY generation) and the other is inside.\nThis exercises the region transition: the walk starts in the\nINFINITY region where side-exhaustion is disabled, then crosses\ninto the finite region where it can fire.\n\nSigned-off-by: Kristofer Karlsson \n---\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 +++++++++++++++++++++++++++\n t/t6600-test-reach.sh | 25 ++++++++\n 3 files changed, 108 insertions(+)\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\ndiff --git a/t/meson.build b/t/meson.build\nindex 3219264fe7..ee6ebdffb9 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -786,6 +786,7 @@ integration_tests = [\n 't6041-bisect-submodule.sh',\n 't6050-replace.sh',\n 't6060-merge-index.sh',\n+ 't6099-merge-base-side-exhaustion.sh',\n 't6100-rev-list-in-order.sh',\n 't6101-rev-parse-parents.sh',\n 't6102-rev-list-unexpected-objects.sh',\ndiff --git a/t/t6099-merge-base-side-exhaustion.sh b/t/t6099-merge-base-side-exhaustion.sh\nnew file mode 100755\nindex 0000000000..4f1e0d50ef\n--- /dev/null\n+++ b/t/t6099-merge-base-side-exhaustion.sh\n@@ -0,0 +1,82 @@\n+#!/bin/sh\n+\n+test_description='merge-base with ancestor among merge-base candidates\n+\n+Test that merge-base --all correctly handles cases where\n+multiple merge-base candidates exist and one is an ancestor\n+of another. The side-exhaustion optimization in\n+paint_down_to_common may exit before STALE propagation\n+removes the ancestor, but remove_redundant catches it.\n+\n+Graph shape (parents are below children):\n+\n+ A ----------- X\n+ |\\ /|\n+ | B---------/ |\n+ | | |\n+ e2 \\ f2\n+ | | |\n+ e1 d1 f1\n+ \\ | /\n+ \\ | /\n+ \\| /\n+ C\n+\n+A and X are the two tips.\n+B and C are both reachable from A and X.\n+B reaches C through d1.\n+Only B should appear in merge-base --all output.\n+'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+TEST_PASSES_SANITIZE_LEAK=true\n+. ./test-lib.sh\n+\n+test_expect_success 'setup ancestor merge-base candidate' '\n+\ttest_commit C &&\n+\n+\tgit checkout -b d-chain HEAD &&\n+\ttest_commit d1 &&\n+\ttest_commit B &&\n+\n+\tgit checkout -b e-path C &&\n+\ttest_commit e1 &&\n+\ttest_commit e2 &&\n+\n+\tgit checkout -b f-path C &&\n+\ttest_commit f1 &&\n+\ttest_commit f2 &&\n+\n+\tgit checkout -b branch-A e-path &&\n+\ttest_merge A B &&\n+\n+\tgit checkout -b branch-X f-path &&\n+\ttest_merge X B &&\n+\n+\tgit commit-graph write --reachable\n+'\n+\n+test_expect_success 'merge-base --all excludes ancestor candidate' '\n+\tgit rev-parse B >expected &&\n+\tgit merge-base --all A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'merge-base (single) finds shallowest' '\n+\tgit rev-parse B >expected &&\n+\tgit merge-base A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+# Without commit-graph: generation numbers are INFINITY,\n+# side-exhaustion optimization does not fire.\n+test_expect_success 'merge-base --all without commit-graph' '\n+\trm -f .git/objects/info/commit-graph &&\n+\tgit rev-parse B >expected &&\n+\tgit merge-base --all A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex c2e091aad1..4b771b4c58 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -294,6 +294,31 @@ test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'setup mixed finite/INFINITY topology' '\n+\t# Create a commit outside all saved commit-graph files so it always\n+\t# has INFINITY generation, while its parent (ps-X) is in the graph\n+\t# with a finite generation. Use the ps-* orphan topology so we do\n+\t# not pollute the grid-based rev-list tests.\n+\tgit checkout ps-X &&\n+\ttest_env GIT_TEST_COMMIT_GRAPH= test_commit pm-INF\n+'\n+\n+test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n+\t# One tip (pm-INF) is outside the commit-graph with INFINITY\n+\t# generation; the other (ps-B) is in the graph with finite\n+\t# generation. The walk starts in the INFINITY region and crosses\n+\t# into the finite region where side-exhaustion can fire.\n+\tcat >input <<-\\EOF &&\n+\tA:pm-INF\n+\tX:ps-B\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse ps-X\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546463","messageId":"2592264cda543c96c4479bb4ba6368c0121e4207.1782479286.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v3 4/8] commit-reach: add trace2 instrumentation to paint_down_to_common()","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:08:01Z","receivedAt":"2026-06-26T13:08:13Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a step counter and trace2_data_intmax() call so that the number\nof commits visited during the paint walk is observable via\nGIT_TRACE2_EVENT. This provides a way to measure the impact of\nfuture optimizations without relying on wall-clock benchmarks alone.\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 5 ++++\n t/t6600-test-reach.sh | 53 ++++++++++++++++++++++++++++++-------------\n 2 files changed, 42 insertions(+), 16 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex a9483759e0..f6a438550b 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -11,6 +11,7 @@\n #include \"tag.h\"\n #include \"commit-reach.h\"\n #include \"ewah/ewok.h\"\n+#include \"trace2.h\"\n \n /* Remember to update object flag allocation in object.h */\n #define PARENT1\t\t(1u<<16)\n@@ -112,6 +113,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t{ compare_commits_by_gen_then_commit_date }\n \t};\n \tint i;\n+\tint steps = 0;\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n@@ -135,6 +137,7 @@ static int paint_down_to_common(struct repository *r,\n \t\tstruct commit_list *parents;\n \t\tint flags;\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n+\t\tsteps++;\n \n \t\tif (min_generation && generation > last_gen)\n \t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n@@ -190,6 +193,8 @@ static int paint_down_to_common(struct repository *r,\n \t}\n \n \tclear_nonstale_queue(&queue);\n+\ttrace2_data_intmax(\"paint_down_to_common\", r,\n+\t\t\t \"steps\", steps);\n \tcommit_list_sort_by_date(result);\n \treturn 0;\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 4b771b4c58..b3a31b80ac 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -118,24 +118,34 @@ test_expect_success 'setup' '\n '\n \n run_all_modes () {\n-\ttest_when_finished rm -rf .git/objects/info/commit-graph &&\n-\t\"$@\" actual &&\n-\ttest_cmp expect actual &&\n-\tcp commit-graph-full .git/objects/info/commit-graph &&\n-\t\"$@\" actual &&\n-\ttest_cmp expect actual &&\n-\tcp commit-graph-half .git/objects/info/commit-graph &&\n-\t\"$@\" actual &&\n-\ttest_cmp expect actual &&\n-\tcp commit-graph-no-gdat .git/objects/info/commit-graph &&\n-\t\"$@\" actual &&\n-\ttest_cmp expect actual\n+\tgraph=.git/objects/info/commit-graph &&\n+\ttest_when_finished rm -rf \"$graph\" \"${graph}s\" &&\n+\trm -f trace-mode-*.txt &&\n+\n+\tfor mode in none full half no-gdat\n+\tdo\n+\t\trm -rf \"$graph\" \"${graph}s\" &&\n+\t\tcp \"commit-graph-${mode}\" \"$graph\" 2>/dev/null ||\n+\t\ttrue &&\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace-mode-${mode}.txt\" \\\n+\t\t\t\"$@\" actual &&\n+\t\ttest_cmp expect actual || return 1\n+\tdone\n }\n \n test_all_modes () {\n \trun_all_modes test-tool reach \"$@\"\n }\n \n+test_paint_down_steps () {\n+\tfor mode in none full half no-gdat\n+\tdo\n+\t\ttest_trace2_data paint_down_to_common steps \"$1\" \\\n+\t\t\t<\"trace-mode-${mode}.txt\" || return 1\n+\t\tshift\n+\tdone\n+}\n+\n test_expect_success 'ref_newer:miss' '\n \tcat >input <<-\\EOF &&\n \tA:commit-5-7\n@@ -209,7 +219,8 @@ test_expect_success 'in_merge_bases_many:self' '\n \tX:commit-6-8\n \tEOF\n \techo \"in_merge_bases_many(A,X):1\" >expect &&\n-\ttest_all_modes in_merge_bases_many\n+\ttest_all_modes in_merge_bases_many &&\n+\ttest_paint_down_steps 45 2 25 3\n '\n \n test_expect_success 'is_descendant_of:hit' '\n@@ -277,7 +288,8 @@ test_expect_success 'get_merge_bases_many:pending-stale' '\n \t\techo \"get_merge_bases_many(A,X):\" &&\n \t\tgit rev-parse ps-B\n \t} >expect &&\n-\ttest_all_modes get_merge_bases_many\n+\ttest_all_modes get_merge_bases_many &&\n+\ttest_paint_down_steps 6 6 6 6\n '\n \n test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n@@ -291,7 +303,8 @@ test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n \t\techo \"get_merge_bases_many(A,X):\" &&\n \t\tgit rev-parse pi-B\n \t} >expect &&\n-\ttest_all_modes get_merge_bases_many\n+\ttest_all_modes get_merge_bases_many &&\n+\ttest_paint_down_steps 5 5 5 5\n '\n \n test_expect_success 'setup mixed finite/INFINITY topology' '\n@@ -316,7 +329,15 @@ test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n \t\techo \"get_merge_bases_many(A,X):\" &&\n \t\tgit rev-parse ps-X\n \t} >expect &&\n-\ttest_all_modes get_merge_bases_many\n+\ttest_all_modes get_merge_bases_many &&\n+\ttest_paint_down_steps 3 3 3 3\n+'\n+\n+test_expect_success 'merge-base --all commit-walk steps' '\n+\t>input &&\n+\tgit rev-parse commit-9-1 >expect &&\n+\trun_all_modes git merge-base --all commit-9-9 commit-9-1 &&\n+\ttest_paint_down_steps 81 80 81 81\n '\n \n test_expect_success 'reduce_heads' '\n-- \ngitgitgadget\n\n"},{"id":"546464","messageId":"e82e0c72b6fc72b214f40efa9586c77790881f93.1782479286.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v3 5/8] commit-reach: introduce struct paint_state with per-side counters","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:08:02Z","receivedAt":"2026-06-26T13:08:15Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a paint_state struct for use by paint_down_to_common() that\nwraps a prio_queue with per-side commit counters. Each non-stale\nqueued commit occupies exactly one counter bucket based on its\npaint flags: PARENT1-only, PARENT2-only, or both sides (a pending\nmerge-base candidate).\n\nThe counters are maintained by paint_count_update() which adjusts\nthe appropriate bucket by a signed delta. An exhaustive switch on\nthe paint+stale bits documents all valid flag combinations in one\nplace.\n\nConvert paint_down_to_common() to use paint_state. The loop now\ndrains the queue via paint_queue_get() which returns NULL when all\ncounters reach zero, replacing the old pointer-based termination\n(max_nonstale). This is equivalent behavior -- both conditions\ndetect that no non-stale entries remain.\n\npaint_queue_get() uses a \"pop first\" form: it dequeues a commit,\nthen checks the counters. This means the loop exits one iteration\nearlier than the old code in some topologies (the popped stale\ncommit is never processed), so a few step counts drop by one.\n\nThe existing nonstale_queue is left in place for ahead_behind().\n\nSigned-off-by: Kristofer Karlsson \n---\n .../technical/paint-down-to-common.adoc | 9 +-\n commit-reach.c | 94 ++++++++++++++++---\n t/t6600-test-reach.sh | 4 +-\n 3 files changed, 85 insertions(+), 22 deletions(-)\n\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nindex c10d5d2887..0f4e1892a5 100644\n--- a/Documentation/technical/paint-down-to-common.adoc\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -88,15 +88,12 @@ re-enqueued is bounded by the number of flag transitions.\n Termination\n -----------\n \n-The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n-tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n-so far. Once that commit is dequeued, every remaining entry is known\n-to be STALE and the loop terminates. Specifically, the main loop\n+The walk tracks the number of commits of each type in the queue\n+(PARENT1-only, PARENT2-only, pending merge-base). The main loop\n ends when one of the following conditions holds:\n \n 1. The queue is empty.\n- 2. `max_nonstale` has been dequeued, meaning the queue only contains\n- STALE entries.\n+ 2. The queue contains only stale entries.\n \n Stale entry condition\n ~~~~~~~~~~~~~~~~~~~~~\ndiff --git a/commit-reach.c b/commit-reach.c\nindex f6a438550b..0f29b143bd 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -97,6 +97,75 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n \treturn commit;\n }\n \n+/*\n+ * Priority queue with per-side commit counters for paint_down_to_common().\n+ * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n+ * PARENT2-only, or both (a pending merge-base candidate).\n+ */\n+struct paint_state {\n+\tstruct prio_queue queue;\n+\tint p1_count;\n+\tint p2_count;\n+\tint pending_merge_bases;\n+};\n+\n+static void paint_count_update(struct paint_state *state,\n+\t\t\t unsigned flags, int delta)\n+{\n+\tswitch (flags & (PARENT1 | PARENT2 | STALE)) {\n+\tcase PARENT1:\n+\t\tstate->p1_count += delta;\n+\t\tbreak;\n+\n+\tcase PARENT2:\n+\t\tstate->p2_count += delta;\n+\t\tbreak;\n+\n+\tcase PARENT1 | PARENT2:\n+\t\tstate->pending_merge_bases += delta;\n+\t\tbreak;\n+\n+\tcase PARENT1 | PARENT2 | STALE:\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tBUG(\"unexpected paint state\");\n+\t}\n+}\n+\n+static void paint_queue_put(struct paint_state *state,\n+\t\t\t struct commit *c, unsigned add_flags)\n+{\n+\tunsigned old_flags = c->object.flags;\n+\tc->object.flags |= add_flags;\n+\n+\tif (old_flags & ENQUEUED) {\n+\t\tpaint_count_update(state, old_flags, -1);\n+\t\tpaint_count_update(state, c->object.flags, 1);\n+\t} else {\n+\t\tc->object.flags |= ENQUEUED;\n+\t\tprio_queue_put(&state->queue, c);\n+\t\tpaint_count_update(state, c->object.flags, 1);\n+\t}\n+}\n+\n+static struct commit *paint_queue_get(struct paint_state *state)\n+{\n+\tstruct commit *commit = prio_queue_get(&state->queue);\n+\n+\tif (!commit)\n+\t\treturn NULL;\n+\n+\tcommit->object.flags &= ~ENQUEUED;\n+\n+\tif (!state->p1_count && !state->p2_count &&\n+\t !state->pending_merge_bases)\n+\t\treturn NULL;\n+\n+\tpaint_count_update(state, commit->object.flags, -1);\n+\treturn commit;\n+}\n+\n /*\n * See Documentation/technical/paint-down-to-common.adoc\n *\n@@ -109,31 +178,29 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\tenum merge_base_flags mb_flags,\n \t\t\t\tstruct commit_list **result)\n {\n-\tstruct nonstale_queue queue = {\n-\t\t{ compare_commits_by_gen_then_commit_date }\n+\tstruct paint_state state = {\n+\t\t.queue = { compare_commits_by_gen_then_commit_date }\n \t};\n+\tstruct commit *commit;\n \tint i;\n \tint steps = 0;\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n \tif (!min_generation && !corrected_commit_dates_enabled(r))\n-\t\tqueue.pq.compare = compare_commits_by_commit_date;\n+\t\tstate.queue.compare = compare_commits_by_commit_date;\n \n \tone->object.flags |= PARENT1;\n \tif (!n) {\n \t\tcommit_list_append(one, result);\n \t\treturn 0;\n \t}\n-\tnonstale_queue_put_dedup(&queue, one);\n+\tpaint_queue_put(&state, one, 0);\n \n-\tfor (i = 0; i < n; i++) {\n-\t\ttwos[i]->object.flags |= PARENT2;\n-\t\tnonstale_queue_put_dedup(&queue, twos[i]);\n-\t}\n+\tfor (i = 0; i < n; i++)\n+\t\tpaint_queue_put(&state, twos[i], PARENT2);\n \n-\twhile (queue.max_nonstale) {\n-\t\tstruct commit *commit = nonstale_queue_get_dedup(&queue);\n+\twhile ((commit = paint_queue_get(&state))) {\n \t\tstruct commit_list *parents;\n \t\tint flags;\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n@@ -172,7 +239,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t\tif ((p->object.flags & flags) == flags)\n \t\t\t\tcontinue;\n \t\t\tif (repo_parse_commit(r, p)) {\n-\t\t\t\tclear_nonstale_queue(&queue);\n+\t\t\t\tclear_prio_queue(&state.queue);\n \t\t\t\tcommit_list_free(*result);\n \t\t\t\t*result = NULL;\n \t\t\t\t/*\n@@ -187,12 +254,11 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\treturn error(_(\"could not parse commit %s\"),\n \t\t\t\t\t oid_to_hex(&p->object.oid));\n \t\t\t}\n-\t\t\tp->object.flags |= flags;\n-\t\t\tnonstale_queue_put_dedup(&queue, p);\n+\t\t\tpaint_queue_put(&state, p, flags);\n \t\t}\n \t}\n \n-\tclear_nonstale_queue(&queue);\n+\tclear_prio_queue(&state.queue);\n \ttrace2_data_intmax(\"paint_down_to_common\", r,\n \t\t\t \"steps\", steps);\n \tcommit_list_sort_by_date(result);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex b3a31b80ac..51f3d70492 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -289,7 +289,7 @@ test_expect_success 'get_merge_bases_many:pending-stale' '\n \t\tgit rev-parse ps-B\n \t} >expect &&\n \ttest_all_modes get_merge_bases_many &&\n-\ttest_paint_down_steps 6 6 6 6\n+\ttest_paint_down_steps 5 5 5 5\n '\n \n test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n@@ -304,7 +304,7 @@ test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n \t\tgit rev-parse pi-B\n \t} >expect &&\n \ttest_all_modes get_merge_bases_many &&\n-\ttest_paint_down_steps 5 5 5 5\n+\ttest_paint_down_steps 5 4 5 5\n '\n \n test_expect_success 'setup mixed finite/INFINITY topology' '\n-- \ngitgitgadget\n\n"},{"id":"546465","messageId":"e6181bf3c108148ad1c6c9f2a62712dfd01bd779.1782479286.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v3 6/8] commit-reach: remove unused nonstale_queue dedup wrappers","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:08:03Z","receivedAt":"2026-06-26T13:08:15Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nnonstale_queue_put_dedup() and nonstale_queue_get_dedup() became\nunused after the previous commit. The core nonstale_queue functions\nremain in use by ahead_behind().\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 18 ------------------\n 1 file changed, 18 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 0f29b143bd..ee0e0fdf6e 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -79,24 +79,6 @@ static void clear_nonstale_queue(struct nonstale_queue *queue)\n \tqueue->max_nonstale = NULL;\n }\n \n-static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n-\t\t\t\t struct commit *c)\n-{\n-\tif (c->object.flags & ENQUEUED)\n-\t\treturn;\n-\tc->object.flags |= ENQUEUED;\n-\tnonstale_queue_put(queue, c);\n-}\n-\n-static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n-{\n-\tstruct commit *commit = nonstale_queue_get(queue);\n-\n-\tif (commit)\n-\t\tcommit->object.flags &= ~ENQUEUED;\n-\treturn commit;\n-}\n-\n /*\n * Priority queue with per-side commit counters for paint_down_to_common().\n * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n-- \ngitgitgadget\n\n"},{"id":"546466","messageId":"f3572a8a89c74fad54a9e53be6f0e34daa2d50c2.1782479286.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v3 7/8] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:08:04Z","receivedAt":"2026-06-26T13:08:16Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd an early termination check to paint_down_to_common() using the\nper-side counters introduced earlier. Once the walk enters the\nfinite-generation region, terminate early when one side's exclusive\ncount drops to zero -- no new merge-base can form without both paint\nsides meeting.\n\nThe check also waits for pending_merge_bases to reach zero, ensuring\nall merge-base candidates have been dequeued and recorded before\nexiting.\n\nThe INFINITY gate ensures correctness: commits without a commit-graph\nentry have GENERATION_NUMBER_INFINITY and are ordered by commit date,\nwhich is not topologically reliable. The optimization only fires\nonce the walk enters the finite-generation region where ordering\nguarantees hold.\n\nWiden the existing generation-monotonicity BUG assertion to fire\nunconditionally, not only when min_generation is set. The\nside-exhaustion optimization depends on correct generation ordering,\nso the assertion should always be active.\n\nStep counts measured with trace2 on git.git with commit-graph:\n\n merge-base --all v2.0.0 v2.55.0-rc1:\n before: 72264 steps after: 44589 steps\n\n merge-base --all v2.55.0-rc1 v2.55.0-rc1~5:\n before: 110 steps after: 7 steps\n\nHelped-by: Derrick Stolee \nHelped-by: Elijah Newren \nSigned-off-by: Kristofer Karlsson \n---\n .../technical/paint-down-to-common.adoc | 17 +++++++++++++++++\n commit-reach.c | 19 +++++++++++++++----\n t/t6600-test-reach.sh | 4 ++--\n 3 files changed, 34 insertions(+), 6 deletions(-)\n\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nindex 0f4e1892a5..983dfcf233 100644\n--- a/Documentation/technical/paint-down-to-common.adoc\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -94,6 +94,9 @@ ends when one of the following conditions holds:\n \n 1. The queue is empty.\n 2. The queue contains only stale entries.\n+ 3. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n+ remain in the queue, no pending merge-base candidates exist,\n+ and the walk has entered the finite-generation region.\n \n Stale entry condition\n ~~~~~~~~~~~~~~~~~~~~~\n@@ -104,6 +107,20 @@ existing candidates by proving one is an ancestor of another, but\n `remove_redundant()` handles that as a post-processing step, so it\n is safe to exit early.\n \n+Side-exhaustion condition\n+~~~~~~~~~~~~~~~~~~~~~~~~~\n+A new merge-base requires commits from both sides to meet. When one\n+side's exclusive counter reaches zero and there are no pending\n+merge-base candidates, no future traversal step can produce a new\n+candidate.\n+\n+This optimization only activates in the finite-generation region\n+where topological ordering holds. In that region, children are\n+always visited before parents, so paint flags are final at visit\n+time and an exhausted side cannot reappear. In the INFINITY region,\n+commit-date ordering can violate this guarantee, so the check is\n+skipped.\n+\n Related documentation\n ---------------------\n \ndiff --git a/commit-reach.c b/commit-reach.c\nindex ee0e0fdf6e..0248d6fedb 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -131,6 +131,10 @@ static void paint_queue_put(struct paint_state *state,\n \t}\n }\n \n+/*\n+ * Dequeue the next commit for the paint walk, or return NULL when\n+ * no more merge bases can be discovered.\n+ */\n static struct commit *paint_queue_get(struct paint_state *state)\n {\n \tstruct commit *commit = prio_queue_get(&state->queue);\n@@ -140,9 +144,16 @@ static struct commit *paint_queue_get(struct paint_state *state)\n \n \tcommit->object.flags &= ~ENQUEUED;\n \n-\tif (!state->p1_count && !state->p2_count &&\n-\t !state->pending_merge_bases)\n-\t\treturn NULL;\n+\tif (!state->pending_merge_bases) {\n+\t\t/* only stale entries remain */\n+\t\tif (!state->p1_count && !state->p2_count)\n+\t\t\treturn NULL;\n+\n+\t\t/* one side is exhausted */\n+\t\tif ((!state->p1_count || !state->p2_count) &&\n+\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n+\t\t\treturn NULL;\n+\t}\n \n \tpaint_count_update(state, commit->object.flags, -1);\n \treturn commit;\n@@ -188,7 +199,7 @@ static int paint_down_to_common(struct repository *r,\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n \t\tsteps++;\n \n-\t\tif (min_generation && generation > last_gen)\n+\t\tif (generation > last_gen)\n \t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n \t\t\t generation, last_gen,\n \t\t\t oid_to_hex(&commit->object.oid));\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 51f3d70492..6365007560 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -220,7 +220,7 @@ test_expect_success 'in_merge_bases_many:self' '\n \tEOF\n \techo \"in_merge_bases_many(A,X):1\" >expect &&\n \ttest_all_modes in_merge_bases_many &&\n-\ttest_paint_down_steps 45 2 25 3\n+\ttest_paint_down_steps 45 1 25 1\n '\n \n test_expect_success 'is_descendant_of:hit' '\n@@ -337,7 +337,7 @@ test_expect_success 'merge-base --all commit-walk steps' '\n \t>input &&\n \tgit rev-parse commit-9-1 >expect &&\n \trun_all_modes git merge-base --all commit-9-9 commit-9-1 &&\n-\ttest_paint_down_steps 81 80 81 81\n+\ttest_paint_down_steps 81 9 57 10\n '\n \n test_expect_success 'reduce_heads' '\n-- \ngitgitgadget\n\n"},{"id":"546467","messageId":"4b9f192d98b8e8f2d30eed4261a73e766eeafcc2.1782479286.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v3 8/8] commit-reach: move min_generation check into paint_queue_get()","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-26T13:08:05Z","receivedAt":"2026-06-26T13:08:17Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nConsolidate the min_generation termination condition into\npaint_queue_get(), alongside the existing stale-entry and\nside-exhaustion checks.\n\nMove last_gen into struct paint_state so that\ncommit_graph_generation() is called exactly once per dequeued commit\nand the result is shared across all termination checks and the\nmonotonicity BUG assertion. The loop body in paint_down_to_common()\nreads state.last_gen instead of recomputing the generation number.\n\nSigned-off-by: Kristofer Karlsson \n---\n .../technical/paint-down-to-common.adoc | 9 ++++++\n commit-reach.c | 31 +++++++++++--------\n 2 files changed, 27 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nindex 983dfcf233..eef249f4a4 100644\n--- a/Documentation/technical/paint-down-to-common.adoc\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -97,6 +97,8 @@ ends when one of the following conditions holds:\n 3. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n remain in the queue, no pending merge-base candidates exist,\n and the walk has entered the finite-generation region.\n+ 4. Generation cutoff: the dequeued commit's generation is below\n+ a caller-supplied `min_generation` threshold.\n \n Stale entry condition\n ~~~~~~~~~~~~~~~~~~~~~\n@@ -121,6 +123,13 @@ time and an exhausted side cannot reappear. In the INFINITY region,\n commit-date ordering can violate this guarantee, so the check is\n skipped.\n \n+Generation cutoff\n+~~~~~~~~~~~~~~~~~\n+Some callers (notably `remove_redundant()`) supply a `min_generation`\n+threshold -- the minimum generation of the input commits. No merge\n+base can have a generation below this threshold, so the walk\n+terminates as soon as it dequeues such a commit.\n+\n Related documentation\n ---------------------\n \ndiff --git a/commit-reach.c b/commit-reach.c\nindex 0248d6fedb..0cd785c31b 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -89,6 +89,8 @@ struct paint_state {\n \tint p1_count;\n \tint p2_count;\n \tint pending_merge_bases;\n+\ttimestamp_t min_generation;\n+\ttimestamp_t last_gen;\n };\n \n static void paint_count_update(struct paint_state *state,\n@@ -138,11 +140,23 @@ static void paint_queue_put(struct paint_state *state,\n static struct commit *paint_queue_get(struct paint_state *state)\n {\n \tstruct commit *commit = prio_queue_get(&state->queue);\n+\ttimestamp_t generation;\n \n \tif (!commit)\n \t\treturn NULL;\n \n \tcommit->object.flags &= ~ENQUEUED;\n+\tgeneration = commit_graph_generation(commit);\n+\n+\tif (generation > state->last_gen)\n+\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n+\t\t generation, state->last_gen,\n+\t\t oid_to_hex(&commit->object.oid));\n+\tstate->last_gen = generation;\n+\n+\t/* generation cutoff */\n+\tif (generation < state->min_generation)\n+\t\treturn NULL;\n \n \tif (!state->pending_merge_bases) {\n \t\t/* only stale entries remain */\n@@ -151,7 +165,7 @@ static struct commit *paint_queue_get(struct paint_state *state)\n \n \t\t/* one side is exhausted */\n \t\tif ((!state->p1_count || !state->p2_count) &&\n-\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n+\t\t generation < GENERATION_NUMBER_INFINITY)\n \t\t\treturn NULL;\n \t}\n \n@@ -177,9 +191,10 @@ static int paint_down_to_common(struct repository *r,\n \tstruct commit *commit;\n \tint i;\n \tint steps = 0;\n-\ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n+\tstate.min_generation = min_generation;\n+\tstate.last_gen = GENERATION_NUMBER_INFINITY;\n \tif (!min_generation && !corrected_commit_dates_enabled(r))\n \t\tstate.queue.compare = compare_commits_by_commit_date;\n \n@@ -196,18 +211,8 @@ static int paint_down_to_common(struct repository *r,\n \twhile ((commit = paint_queue_get(&state))) {\n \t\tstruct commit_list *parents;\n \t\tint flags;\n-\t\ttimestamp_t generation = commit_graph_generation(commit);\n \t\tsteps++;\n \n-\t\tif (generation > last_gen)\n-\t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n-\t\t\t generation, last_gen,\n-\t\t\t oid_to_hex(&commit->object.oid));\n-\t\tlast_gen = generation;\n-\n-\t\tif (generation < min_generation)\n-\t\t\tbreak;\n-\n \t\tflags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n \t\tif (flags == (PARENT1 | PARENT2)) {\n \t\t\tif (!(commit->object.flags & RESULT)) {\n@@ -219,7 +224,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\t * descendant of this one.\n \t\t\t\t */\n \t\t\t\tif (!(mb_flags & MERGE_BASE_FIND_ALL) &&\n-\t\t\t\t generation < GENERATION_NUMBER_INFINITY)\n+\t\t\t\t state.last_gen < GENERATION_NUMBER_INFINITY)\n \t\t\t\t\tbreak;\n \t\t\t}\n \t\t\t/* Mark parents of a found merge stale */\n-- \ngitgitgadget\n"},{"id":"546471","messageId":"CAL71e4N3RPHSrXscwYJUiLWc8-a172h+nE13yuUBRV7Uu3zGzw@mail.gmail.com","threadId":"65846","inReplyTo":"f3572a8a89c74fad54a9e53be6f0e34daa2d50c2.1782479286.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 7/8] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-26T14:29:12Z","receivedAt":"2026-06-26T14:29:24Z","isPatch":true,"body":"On Fri, 26 Jun 2026 at 15:08, Kristofer Karlsson via GitGitGadget\n wrote:\n>\n> From: Kristofer Karlsson \n>\n> - if (min_generation && generation > last_gen)\n> + if (generation > last_gen)\n\nI have to note that I accidentally pushed this version before noticing\nthat it now fails for a subset of commit-graph modes.\nApologies for that - I will rework the logic here later\nto preserve the behavior better.\n\nI think (and hope) the rest of the patch series is in good shape though\nand addressed the previous feedback, so any partial new review\nfeedback would still be appreciated.\n\nThanks,\nKristofer\n"},{"id":"546472","messageId":"a74d3114-7d7f-469a-b181-60853bb82864@gmail.com","threadId":"65846","inReplyTo":"2592264cda543c96c4479bb4ba6368c0121e4207.1782479286.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 4/8] commit-reach: add trace2 instrumentation to paint_down_to_common()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-26T14:31:09Z","receivedAt":"2026-06-26T14:31:12Z","isPatch":true,"body":"On 6/26/2026 9:08 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n\n> run_all_modes () {\n> -\ttest_when_finished rm -rf .git/objects/info/commit-graph &&\n> -\t\"$@\" actual &&\n> -\ttest_cmp expect actual &&\n> -\tcp commit-graph-full .git/objects/info/commit-graph &&\n> -\t\"$@\" actual &&\n> -\ttest_cmp expect actual &&\n> -\tcp commit-graph-half .git/objects/info/commit-graph &&\n> -\t\"$@\" actual &&\n> -\ttest_cmp expect actual &&\n> -\tcp commit-graph-no-gdat .git/objects/info/commit-graph &&\n> -\t\"$@\" actual &&\n> -\ttest_cmp expect actual\n> +\tgraph=.git/objects/info/commit-graph &&\n> +\ttest_when_finished rm -rf \"$graph\" \"${graph}s\" &&\n> +\trm -f trace-mode-*.txt &&\n> +\n> +\tfor mode in none full half no-gdat\n> +\tdo\n> +\t\trm -rf \"$graph\" \"${graph}s\" &&\n> +\t\tcp \"commit-graph-${mode}\" \"$graph\" 2>/dev/null ||\n> +\t\ttrue &&\n> +\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace-mode-${mode}.txt\" \\\n> +\t\t\t\"$@\" actual &&\n> +\t\ttest_cmp expect actual || return 1\n> +\tdone\n> }\n\nThank you for putting these traces into this helper AND for\nmaking it cleaner at the same time!\n\n> +test_paint_down_steps () {\n> +\tfor mode in none full half no-gdat\n> +\tdo\n> +\t\ttest_trace2_data paint_down_to_common steps \"$1\" \\\n> +\t\t\t<\"trace-mode-${mode}.txt\" || return 1\n> +\t\tshift\n> +\tdone\n> +}\n> +\n> test_expect_success 'ref_newer:miss' '\n> \tcat >input <<-\\EOF &&\n> \tA:commit-5-7\n> @@ -209,7 +219,8 @@ test_expect_success 'in_merge_bases_many:self' '\n> \tX:commit-6-8\n> \tEOF\n> \techo \"in_merge_bases_many(A,X):1\" >expect &&\n> -\ttest_all_modes in_merge_bases_many\n> +\ttest_all_modes in_merge_bases_many &&\n> +\ttest_paint_down_steps 45 2 25 3\n> '\n\noooh that's clean. Thanks!\n\nWay to over-achieve here. Thanks for going the extra mile with\nthis patch.\n\nThanks,\n-Stolee\n\n"},{"id":"546473","messageId":"cb82cb80-291b-4a36-ae83-f308560c844b@gmail.com","threadId":"65846","inReplyTo":"CAL71e4N3RPHSrXscwYJUiLWc8-a172h+nE13yuUBRV7Uu3zGzw@mail.gmail.com","subject":"Re: [PATCH v3 7/8] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-26T14:32:50Z","receivedAt":"2026-06-26T14:32:52Z","isPatch":true,"body":"On 6/26/2026 10:29 AM, Kristofer Karlsson wrote:\n> On Fri, 26 Jun 2026 at 15:08, Kristofer Karlsson via GitGitGadget\n> wrote:\n>>\n>> From: Kristofer Karlsson \n>>\n>> - if (min_generation && generation > last_gen)\n>> + if (generation > last_gen)\n> \n> I have to note that I accidentally pushed this version before noticing\n> that it now fails for a subset of commit-graph modes.\n> Apologies for that - I will rework the logic here later\n> to preserve the behavior better.\n\nAnd do we catch this with a test case? I'm hoping that you discovered\nthis error through the test suite, even if you submitted the series a\nlittle early. \n> I think (and hope) the rest of the patch series is in good shape though\n> and addressed the previous feedback, so any partial new review\n> feedback would still be appreciated.\nThanks for calling this out, as now I can avoid trying to understand\nthis change during my review.\n\nThanks,\n-Stolee\n\n"},{"id":"546474","messageId":"5edd5912-80b2-4372-b921-52c20e496276@gmail.com","threadId":"65846","inReplyTo":"f3572a8a89c74fad54a9e53be6f0e34daa2d50c2.1782479286.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 7/8] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-26T14:35:17Z","receivedAt":"2026-06-26T14:35:20Z","isPatch":true,"body":"On 6/26/2026 9:08 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n\n> @@ -140,9 +144,16 @@ static struct commit *paint_queue_get(struct paint_state *state)\n> \n> \tcommit->object.flags &= ~ENQUEUED;\n> \n> -\tif (!state->p1_count && !state->p2_count &&\n> -\t !state->pending_merge_bases)\n> -\t\treturn NULL;\n> +\tif (!state->pending_merge_bases) {\n> +\t\t/* only stale entries remain */\n> +\t\tif (!state->p1_count && !state->p2_count)\n> +\t\t\treturn NULL;\n> +\n> +\t\t/* one side is exhausted */\n> +\t\tif ((!state->p1_count || !state->p2_count) &&\n> +\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n> +\t\t\treturn NULL;\n> +\t}\n\nThis continues to look correct. \n> \tpaint_count_update(state, commit->object.flags, -1);\n> \treturn commit;\n> @@ -188,7 +199,7 @@ static int paint_down_to_common(struct repository *r,\n> \t\ttimestamp_t generation = commit_graph_generation(commit);\n> \t\tsteps++;\n> \n> -\t\tif (min_generation && generation > last_gen)\n> +\t\tif (generation > last_gen)\n> \t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n> \t\t\t generation, last_gen,\n> \t\t\t oid_to_hex(&commit->object.oid));\n\nYou mention in your own reply that this is broken. This also looks\nlike a stray change for this patch, so perhaps your end state is\ncorrect despite this patch causing failures. Will inspect soon.\n\n> -\ttest_paint_down_steps 45 2 25 3\n> +\ttest_paint_down_steps 45 1 25 1\n...> -\ttest_paint_down_steps 81 80 81 81\n> +\ttest_paint_down_steps 81 9 57 10\nThese diffs are satisfying.\n\nThanks,\n-Stolee\n\n"},{"id":"546475","messageId":"CAL71e4Oc2eUxSjF9wW=JbAqt8kVoiuDzkDHysGorHC_WEFu9fw@mail.gmail.com","threadId":"65846","inReplyTo":"a74d3114-7d7f-469a-b181-60853bb82864@gmail.com","subject":"Re: [PATCH v3 4/8] commit-reach: add trace2 instrumentation to paint_down_to_common()","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-26T14:35:11Z","receivedAt":"2026-06-26T14:35:24Z","isPatch":true,"body":"On Fri, 26 Jun 2026 at 16:31, Derrick Stolee wrote:\n>\n> > test_expect_success 'ref_newer:miss' '\n> > cat >input <<-\\EOF &&\n> > A:commit-5-7\n> > @@ -209,7 +219,8 @@ test_expect_success 'in_merge_bases_many:self' '\n> > X:commit-6-8\n> > EOF\n> > echo \"in_merge_bases_many(A,X):1\" >expect &&\n> > - test_all_modes in_merge_bases_many\n> > + test_all_modes in_merge_bases_many &&\n> > + test_paint_down_steps 45 2 25 3\n> > '\n>\n> oooh that's clean. Thanks!\n>\n> Way to over-achieve here. Thanks for going the extra mile with\n> this patch.\n\nThanks! I did not want to change it too much but this felt like\na natural place to simplify it a bit.\n\nI also have a local branch now for adding a\ntest_trace2_data_singular helper function that provides\nmore diagnostics on failures\n(show expected vs actual similar to test_cmp)\nbut I'll submit that separately later to limit the scope creep here.\n\nThanks,\nKristofer\n"},{"id":"546477","messageId":"CAL71e4ON=rewXC+xnWHYg-tmWAW6KZ2mp8tKMKyv1Cr3ZePHLQ@mail.gmail.com","threadId":"65846","inReplyTo":"5edd5912-80b2-4372-b921-52c20e496276@gmail.com","subject":"Re: [PATCH v3 7/8] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-26T14:39:51Z","receivedAt":"2026-06-26T14:40:04Z","isPatch":true,"body":"On Fri, 26 Jun 2026 at 16:35, Derrick Stolee wrote:\n>\n> > - if (min_generation && generation > last_gen)\n> > + if (generation > last_gen)\n> > BUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n> > generation, last_gen,\n> > oid_to_hex(&commit->object.oid));\n>\n> You mention in your own reply that this is broken. This also looks\n> like a stray change for this patch, so perhaps your end state is\n> correct despite this patch causing failures. Will inspect soon.\n\nI did not intend it to be a stray change, but rather a natural followup\nto the idea that we could fold all of the halt conditions into the same\nplace. I am happy to either revert that part for v4 (to keep the change\nsimpler, but not fully unified) or fix it properly - I think it should be easy\nsince this was just human error, not a sign of a fundamentally tricky\nproblem.\n\n> > - test_paint_down_steps 45 2 25 3\n> > + test_paint_down_steps 45 1 25 1\n> ...> - test_paint_down_steps 81 80 81 81\n> > + test_paint_down_steps 81 9 57 10\n> These diffs are satisfying.\n\nAgreed! It was nice to introduce the steps counter to the\ntest suite, showing that the patch reached its intended goal\nwhich is clearer than just having benchmarks in the messages.\n\nThanks again,\nKristofer\n"},{"id":"546478","messageId":"34ff8be2-1b3c-480f-ae27-9d65875e6e62@gmail.com","threadId":"65846","inReplyTo":"4b9f192d98b8e8f2d30eed4261a73e766eeafcc2.1782479286.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 8/8] commit-reach: move min_generation check into paint_queue_get()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-26T14:42:55Z","receivedAt":"2026-06-26T14:42:57Z","isPatch":true,"body":"On 6/26/2026 9:08 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n> \n> Consolidate the min_generation termination condition into\n> paint_queue_get(), alongside the existing stale-entry and\n> side-exhaustion checks.\n> \n> Move last_gen into struct paint_state so that\n> commit_graph_generation() is called exactly once per dequeued commit\n> and the result is shared across all termination checks and the\n> monotonicity BUG assertion. The loop body in paint_down_to_common()\n> reads state.last_gen instead of recomputing the generation number.\n\nThanks for incorporating this change into this version.\n\n> + 4. Generation cutoff: the dequeued commit's generation is below\n> + a caller-supplied `min_generation` threshold.\n\nTechnically, this was always a termination condition of the walk,\nbut now we are correcting the documentation to match. It was just\nnot part of the termination in the dequeue method until now.\n\n> @@ -89,6 +89,8 @@ struct paint_state {\n> \tint p1_count;\n> \tint p2_count;\n> \tint pending_merge_bases;\n> +\ttimestamp_t min_generation;\n> +\ttimestamp_t last_gen;\n> };\n\nI'm happy that these details are being imported into the struct.\n\nMy first reaction is that last_gen shouldn't be here because we\ncan see a generation from the dequeued commit. I'll read on to\nbe sure.\n \n> static void paint_count_update(struct paint_state *state,\n> @@ -138,11 +140,23 @@ static void paint_queue_put(struct paint_state *state,\n> static struct commit *paint_queue_get(struct paint_state *state)\n> {\n> \tstruct commit *commit = prio_queue_get(&state->queue);\n> +\ttimestamp_t generation;\n> \n> \tif (!commit)\n> \t\treturn NULL;\n> \n> \tcommit->object.flags &= ~ENQUEUED;\n> +\tgeneration = commit_graph_generation(commit);\n> +\n> +\tif (generation > state->last_gen)\n> +\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n> +\t\t generation, state->last_gen,\n> +\t\t oid_to_hex(&commit->object.oid));\n\nOh I see. It's just for this condition.\n\nDoes this case still break without 'state->min_generation' in the\ncondition?\n\n> +\tstate->last_gen = generation;\n\nThis is an appropriate use of this value. My concerns are no longer\nvalid. Thanks for letting me think out loud.\n\n> +\t/* generation cutoff */\n> +\tif (generation < state->min_generation)\n> +\t\treturn NULL;\n\nAnd here's the crux. Again, impossible for this to halt when\nmin_generation is zero.\n\n> \tif (!state->pending_merge_bases) {\n> \t\t/* only stale entries remain */\n> @@ -151,7 +165,7 @@ static struct commit *paint_queue_get(struct paint_state *state)\n> \n> \t\t/* one side is exhausted */\n> \t\tif ((!state->p1_count || !state->p2_count) &&\n> -\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n> +\t\t generation < GENERATION_NUMBER_INFINITY)\n> \t\t\treturn NULL;\n\nGood reuse of the value.\n\n> \t}\n> \n> @@ -177,9 +191,10 @@ static int paint_down_to_common(struct repository *r,\n> \tstruct commit *commit;\n> \tint i;\n> \tint steps = 0;\n> -\ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n> \tstruct commit_list **tail = result;\n> \n> +\tstate.min_generation = min_generation;\n> +\tstate.last_gen = GENERATION_NUMBER_INFINITY;\n> \tif (!min_generation && !corrected_commit_dates_enabled(r))\n> \t\tstate.queue.compare = compare_commits_by_commit_date;\n> \n> @@ -196,18 +211,8 @@ static int paint_down_to_common(struct repository *r,\n> \twhile ((commit = paint_queue_get(&state))) {\n> \t\tstruct commit_list *parents;\n> \t\tint flags;\n> -\t\ttimestamp_t generation = commit_graph_generation(commit);\n> \t\tsteps++;\n> \n> -\t\tif (generation > last_gen)\n> -\t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n> -\t\t\t generation, last_gen,\n> -\t\t\t oid_to_hex(&commit->object.oid));\n> -\t\tlast_gen = generation;\n> -\n> -\t\tif (generation < min_generation)\n> -\t\t\tbreak;\n> -\n\nI'm happy this is getting cleaned up.\n\n> \t\tflags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n> \t\tif (flags == (PARENT1 | PARENT2)) {\n> \t\t\tif (!(commit->object.flags & RESULT)) {\n> @@ -219,7 +224,7 @@ static int paint_down_to_common(struct repository *r,\n> \t\t\t\t * descendant of this one.\n> \t\t\t\t */\n> \t\t\t\tif (!(mb_flags & MERGE_BASE_FIND_ALL) &&\n> -\t\t\t\t generation < GENERATION_NUMBER_INFINITY)\n> +\t\t\t\t state.last_gen < GENERATION_NUMBER_INFINITY)\n> \t\t\t\t\tbreak;\n> \t\t\t}\n> \t\t\t/* Mark parents of a found merge stale */\n\nAnd here's another termination condition. We are now leaking the\nabstraction of the 'state.last_gen' which give me some bad feelings.\n\nWe are getting to the point where I'd leave such a thing for a\nfollow-up, but since you are needing to re-roll, then this is\nanother case where we can move this into the paint_queue_get(). I\ndon't think this is me \"raising the bar\" from earlier recommendations,\nbecause I was asking for all loop termination to be in the get()\nmethod, if possible.\n\nBut also: I'm not looking at the full method right now to see if\nterminating _at this location in the loop_ is critical. So it may\nvery well be impossible to move this into the get() call, in which\ncase please ignore this suggestion and use state.last_gen.\n\nThanks,\n-Stolee\n\n"},{"id":"546479","messageId":"CAL71e4PWmVjh5pQATGj1GrwgtWDZOeawKUXbKZ7DZX-DcWuCfw@mail.gmail.com","threadId":"65846","inReplyTo":"34ff8be2-1b3c-480f-ae27-9d65875e6e62@gmail.com","subject":"Re: [PATCH v3 8/8] commit-reach: move min_generation check into paint_queue_get()","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-26T14:53:03Z","receivedAt":"2026-06-26T14:53:15Z","isPatch":true,"body":"On Fri, 26 Jun 2026 at 16:42, Derrick Stolee wrote:\n>\n> > + 4. Generation cutoff: the dequeued commit's generation is below\n> > + a caller-supplied `min_generation` threshold.\n>\n> Technically, this was always a termination condition of the walk,\n> but now we are correcting the documentation to match. It was just\n> not part of the termination in the dequeue method until now.\n\nYou're right, I should perhaps fold it into the first patch instead,\nwhich would be logically more accurate. Would be an easy thing\nto fix for a v4.\n\n> > flags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n> > if (flags == (PARENT1 | PARENT2)) {\n> > if (!(commit->object.flags & RESULT)) {\n> > @@ -219,7 +224,7 @@ static int paint_down_to_common(struct repository *r,\n> > * descendant of this one.\n> > */\n> > if (!(mb_flags & MERGE_BASE_FIND_ALL) &&\n> > - generation < GENERATION_NUMBER_INFINITY)\n> > + state.last_gen < GENERATION_NUMBER_INFINITY)\n> > break;\n> > }\n> > /* Mark parents of a found merge stale */\n>\n> And here's another termination condition. We are now leaking the\n> abstraction of the 'state.last_gen' which give me some bad feelings.\n\nYes, this is one of the minor annoyances I also noticed,\nbut it's not too bad. I think a followup could be to either:\n\n1. remove this optimization entirely (though I will have to spend\nsome time reasoning if there are realistic use cases where this\nwould trigger much earlier than side exhaustion.\n\n2. tweak the logic to instead halting on exactly this commit,\ninstead halt inside paint_queue_get if:\n generation < INFINITY && !FIND_ALL && num_results >= 1\nThis would change the semantics slightly (but for the better?)\nin the the found merge-base could be in the infinite region but\nnear the finite region and thus would unlock the optimization\nas soon as we pass that boundary. But I did not want to include\nthat change in this series, which is perhaps already getting\ntoo complex.\n\n> We are getting to the point where I'd leave such a thing for a\n> follow-up, but since you are needing to re-roll, then this is\n> another case where we can move this into the paint_queue_get(). I\n> don't think this is me \"raising the bar\" from earlier recommendations,\n> because I was asking for all loop termination to be in the get()\n> method, if possible.\n>\n> But also: I'm not looking at the full method right now to see if\n> terminating _at this location in the loop_ is critical. So it may\n> very well be impossible to move this into the get() call, in which\n> case please ignore this suggestion and use state.last_gen.\n\nI think it's not critical (as I mentioned above) and I think I will\nneed to follow up on this later.\n\nThanks,\nKristofer\n"},{"id":"546480","messageId":"7069f67b-000c-4b23-9278-e52f571ce6ed@gmail.com","threadId":"65846","inReplyTo":"CAL71e4PWmVjh5pQATGj1GrwgtWDZOeawKUXbKZ7DZX-DcWuCfw@mail.gmail.com","subject":"Re: [PATCH v3 8/8] commit-reach: move min_generation check into paint_queue_get()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-26T14:58:04Z","receivedAt":"2026-06-26T14:58:07Z","isPatch":true,"body":"On 6/26/2026 10:53 AM, Kristofer Karlsson wrote:\n> On Fri, 26 Jun 2026 at 16:42, Derrick Stolee wrote:\n>>\n>>> + 4. Generation cutoff: the dequeued commit's generation is below\n>>> + a caller-supplied `min_generation` threshold.\n>>\n>> Technically, this was always a termination condition of the walk,\n>> but now we are correcting the documentation to match. It was just\n>> not part of the termination in the dequeue method until now.\n> \n> You're right, I should perhaps fold it into the first patch instead,\n> which would be logically more accurate. Would be an easy thing\n> to fix for a v4.\n\nAnd the remaining condition exposed in this diff isn't included,\neither:\n \n>>> flags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n>>> if (flags == (PARENT1 | PARENT2)) {\n>>> if (!(commit->object.flags & RESULT)) {\n>>> @@ -219,7 +224,7 @@ static int paint_down_to_common(struct repository *r,\n>>> * descendant of this one.\n>>> */\n>>> if (!(mb_flags & MERGE_BASE_FIND_ALL) &&\n>>> - generation < GENERATION_NUMBER_INFINITY)\n>>> + state.last_gen < GENERATION_NUMBER_INFINITY)\n>>> break;\n>>> }\n>>> /* Mark parents of a found merge stale */\n>>\n>> And here's another termination condition. We are now leaking the\n>> abstraction of the 'state.last_gen' which give me some bad feelings.\n> \n> Yes, this is one of the minor annoyances I also noticed,\n> but it's not too bad. I think a followup could be to either:\n> \n> 1. remove this optimization entirely (though I will have to spend\n> some time reasoning if there are realistic use cases where this\n> would trigger much earlier than side exhaustion.\n> \n> 2. tweak the logic to instead halting on exactly this commit,\n> instead halt inside paint_queue_get if:\n> generation < INFINITY && !FIND_ALL && num_results >= 1\n> This would change the semantics slightly (but for the better?)\n> in the the found merge-base could be in the infinite region but\n> near the finite region and thus would unlock the optimization\n> as soon as we pass that boundary. But I did not want to include\n> that change in this series, which is perhaps already getting\n> too complex.\n\nI'm happy to keep this one out of the series. Perhaps it would\nbe good for you to finish this series with the current scope\nand then for another contributor (me, probably) to do another\nround of cleanup/reaction on top. I only say \"another\ncontributor\" because new eyes can help to see new things, outside\nof a patch diff.\n\n>> We are getting to the point where I'd leave such a thing for a\n>> follow-up, but since you are needing to re-roll, then this is\n>> another case where we can move this into the paint_queue_get(). I\n>> don't think this is me \"raising the bar\" from earlier recommendations,\n>> because I was asking for all loop termination to be in the get()\n>> method, if possible.\n>>\n>> But also: I'm not looking at the full method right now to see if\n>> terminating _at this location in the loop_ is critical. So it may\n>> very well be impossible to move this into the get() call, in which\n>> case please ignore this suggestion and use state.last_gen.\n> \n> I think it's not critical (as I mentioned above) and I think I will\n> need to follow up on this later.\nSounds good.\n-Stolee\n\n"},{"id":"546486","messageId":"xmqqjyrl8di0.fsf@gitster.g","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-06-26T16:36:55Z","receivedAt":"2026-06-26T16:36:57Z","isPatch":true,"body":"\"Kristofer Karlsson via GitGitGadget\" \nwrites:\n\n> Changes since v2:\n>\n> * New patch 8/8: moved the min_generation termination check and the\n> last_gen monotonicity assertion into paint_queue_get(), consolidating\n> halt conditions. commit_graph_generation() is now called once per\n> dequeued commit and shared across all checks.\n>\n> * Widened the generation-monotonicity BUG assertion to fire\n> unconditionally, not only when min_generation is set. The side-exhaustion\n> optimization depends on correct generation ordering, so the assertion\n> should always be active. This is a behavior change: the BUG() now fires\n> for any generation ordering violation, regardless of the caller.\n>\n> * Moved all halt conditions inside paint_queue_get() with the \"pop first\"\n> form: pop, check, then decrement counters. This keeps the optimization\n> commit's diff minimal (just inserting the new checks between pop and\n> decrement).\n>\n> * Shortened the doc comment on paint_queue_get() to describe what it does\n> rather than how. Inline comments on each return NULL explain the specific\n> halt condition.\n>\n> * Replaced the manual commit-graph setup in the step-count test with\n> run_all_modes, which now sets GIT_TRACE2_EVENT per mode and produces\n> trace-mode-{none,full,half,no-gdat}.txt files.\n>\n> * Added a test_paint_down_steps helper for concise 4-mode step assertions\n> with diagnostic output on mismatch (prints \"expected X, got Y\" instead of\n> a silent grep failure).\n>\n> * Added step-count assertions to the single-walk edge-case tests:\n> in_merge_bases_many:self, pending-stale, infinity-both-sides,\n> mixed-finite-infinity.\n>\n> * Included step counts alongside wall-clock times in the benchmark tables.\n\nI am getting this failure standalone, when applied on the same base\nas where v2 was applied earlier. For now I'll eject it from 'seen'.\n\nexpecting success of 6600.12 'get_merge_bases_many': \n cat >input <<-\\EOF &&\n A:commit-5-7\n X:commit-4-8\n X:commit-6-6\n X:commit-8-3\n EOF\n {\n echo \"get_merge_bases_many(A,X):\" &&\n git rev-parse commit-5-6 \\\n commit-4-7 | sort\n } >expect &&\n test_all_modes get_merge_bases_many\n\nBUG: commit-reach.c:152: bad generation skip 9 > 8 at 772be737f220103f706a39013c0d115009feefec\nnot ok 12 - get_merge_bases_many\n#\n# cat >input <<-\\EOF &&\n# A:commit-5-7\n# X:commit-4-8\n"},{"id":"546488","messageId":"CAL71e4PcCGbth+CcU0ievjQeK_VnDVwHX=FEvR5jpSkzkr_6GA@mail.gmail.com","threadId":"65846","inReplyTo":"cb82cb80-291b-4a36-ae83-f308560c844b@gmail.com","subject":"Re: [PATCH v3 7/8] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-26T16:41:18Z","receivedAt":"2026-06-26T16:41:30Z","isPatch":true,"body":"On Fri, 26 Jun 2026 at 16:32, Derrick Stolee wrote:\n>\n> > I have to note that I accidentally pushed this version before noticing\n> > that it now fails for a subset of commit-graph modes.\n> > Apologies for that - I will rework the logic here later\n> > to preserve the behavior better.\n>\n> And do we catch this with a test case? I'm hoping that you discovered\n> this error through the test suite, even if you submitted the series a\n> little early.\n\n(I missed replying to this message initially, sorry)\n\nYes exactly - it was caught by t6600 but somehow I missed running it before\nsubmitting it (so I noticed it in the GGG CI instead)\n\nSo the existing tests are good, I only wish I could be equally reliable as them.\n\nThanks,\nKristofer\n"},{"id":"546490","messageId":"CAL71e4OT0brcmbNXBzKpZuxTh3=R0j+zgxWmV4S-weT3q=vpvQ@mail.gmail.com","threadId":"65846","inReplyTo":"xmqqjyrl8di0.fsf@gitster.g","subject":"Re: [PATCH v3 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-26T16:43:58Z","receivedAt":"2026-06-26T16:44:10Z","isPatch":true,"body":"On Fri, 26 Jun 2026 at 18:36, Junio C Hamano wrote:\n>\n> I am getting this failure standalone, when applied on the same base\n> as where v2 was applied earlier. For now I'll eject it from 'seen'.\n>\n\nYes, I am sorry about that. I submitted it after having missed running t6600\nand accidentally having introduced a bug. I tried to self-report it to avoid\nwasting your time, but I only did so in the relevant v3 patch and\nnot this main message.\n\nI will be more careful for v4 (and onwards)\n\nThanks,\nKristofer\n"},{"id":"546497","messageId":"xmqq1pdt87mb.fsf@gitster.g","threadId":"65846","inReplyTo":"CAL71e4OT0brcmbNXBzKpZuxTh3=R0j+zgxWmV4S-weT3q=vpvQ@mail.gmail.com","subject":"Re: [PATCH v3 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-06-26T18:43:56Z","receivedAt":"2026-06-26T18:43:59Z","isPatch":true,"body":"Kristofer Karlsson writes:\n\n> On Fri, 26 Jun 2026 at 18:36, Junio C Hamano wrote:\n>>\n>> I am getting this failure standalone, when applied on the same base\n>> as where v2 was applied earlier. For now I'll eject it from 'seen'.\n>>\n>\n> Yes, I am sorry about that. I submitted it after having missed running t6600\n> and accidentally having introduced a bug. I tried to self-report it to avoid\n> wasting your time, but I only did so in the relevant v3 patch and\n> not this main message.\n\nAh, I see. Mistakes happen, and do not need to rush, as\ncollaboration is asynchronous around here anyway, and we may read\nour e-mails in different order ;-)\n\nThanks. It would have been a more troubling experience if only my\nset-up were seeing the issue, but I am glad that is not the case.\n\n"},{"id":"546513","messageId":"bd37b80d-9eff-496d-8f1f-436594968678@web.de","threadId":"65846","inReplyTo":"e82e0c72b6fc72b214f40efa9586c77790881f93.1782479286.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v3 5/8] commit-reach: introduce struct paint_state with per-side counters","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2026-06-26T21:13:32Z","receivedAt":"2026-06-26T21:13:35Z","isPatch":true,"body":"On 6/26/26 3:08 PM, Kristofer Karlsson via GitGitGadget wrote:\n> \n> diff --git a/commit-reach.c b/commit-reach.c\n> index f6a438550b..0f29b143bd 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -97,6 +97,75 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n> \treturn commit;\n> }\n> \n> +/*\n> + * Priority queue with per-side commit counters for paint_down_to_common().\n> + * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n> + * PARENT2-only, or both (a pending merge-base candidate).\n> + */\n> +struct paint_state {\n> +\tstruct prio_queue queue;\n> +\tint p1_count;\n> +\tint p2_count;\n> +\tint pending_merge_bases;\n> +};\nCan they become negative? Wouldn't size_t be a more natural fit,\nmatching nr from struct prio_queue?\n\nAnd some bikeshedding:\n\nWhy abbreviate? parent1_count and parent2_count would be slightly\neasier to read and associate with PARENT1 and PARENT2.\n\nAnd pending_merge_bases is a counter as well. Why not call it\nlike that, pending_merge_base_count? Well, that's pretty long.\nboth_count? That's quite generic and nondescript. Call the other\ncounters parents1 and parents2? Nah. Or parent1s and parent2s?\nNot sure why this inconsistency bothers me to begin with.\n\nRené\n\n"},{"id":"546518","messageId":"CAL71e4O470P7i55C1yHyS9zjmDw2fY8J19KbywFgQkfvenBe1g@mail.gmail.com","threadId":"65846","inReplyTo":"bd37b80d-9eff-496d-8f1f-436594968678@web.de","subject":"Re: [PATCH v3 5/8] commit-reach: introduce struct paint_state with per-side counters","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-26T21:57:37Z","receivedAt":"2026-06-26T21:57:50Z","isPatch":true,"body":"On Fri, 26 Jun 2026 at 23:13, René Scharfe wrote:\n>\n> > +struct paint_state {\n> > + struct prio_queue queue;\n> > + int p1_count;\n> > + int p2_count;\n> > + int pending_merge_bases;\n> > +};\n> Can they become negative? Wouldn't size_t be a more natural fit,\n> matching nr from struct prio_queue?\n\nNegative would be a clear indication of a bug though that's\nnot checked right now anyway. And since it's not checked\nwe might as well use size_t instead - and it would technically\nbe more correct though I struggle to imagine a case where\nthe number of active elements in the frontier exceeds 2^31\nor whatever a signed int would give.\n\nI am happy to change to size_t.\n\n> And some bikeshedding:\n>\n> Why abbreviate? parent1_count and parent2_count would be slightly\n> easier to read and associate with PARENT1 and PARENT2.\n>\n> And pending_merge_bases is a counter as well. Why not call it\n> like that, pending_merge_base_count? Well, that's pretty long.\n> both_count? That's quite generic and nondescript. Call the other\n> counters parents1 and parents2? Nah. Or parent1s and parent2s?\n> Not sure why this inconsistency bothers me to begin with.\n\nFair point, I was thinking that the surrounding context is so small\nthat the naming almost doesn't matter - the terms don't\nescape paint_down_to_common.\n\nI am happy to change to something like:\nparent1_count, parent2_count, mb_candidate_count\nto make it more consistent.\n\nIt seems the mb_ prefix is already used for\nmerge bases in some files - best example is perhaps builtin/diff.c\n\nI see in the codebase that we are using multiple styles,\nperhaps depending on specific context.\n\n- nr_ prefix: nr_objects, nr_paths_watching\n- num_ prefix: num_commits, num_hashes, num_workers\n- _count suffix: entry_count, max_count, skip_count\n\nso I think _count suffix is a good choice at least - it matches\nother usages where we typically just increment or decrement.\n\nThanks,\nKristofer\n"},{"id":"546600","messageId":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v3.git.1782479286.gitgitgadget@gmail.com","subject":"[PATCH v4 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:38Z","receivedAt":"2026-06-28T12:25:51Z","isPatch":true,"body":"commit-reach: terminate merge-base walk when one paint side is exhausted\n\nOptimize paint_down_to_common() for merge-base queries that hit large\none-sided histories.\n\nWhen the walk from one side reaches a commit with a very low generation\nnumber that the other side never paints, the walk is forced to drain most of\nthe graph. A common trigger is a repository import that grafts a separate\nhistory with its own root, but any merge that introduces a low-generation\ncommit never painted by the other side has the same effect.\n\nA new merge-base candidate can only be discovered when exclusive PARENT1 and\nPARENT2 paint meet. This series teaches paint_down_to_common() to stop as\nsoon as one side has no exclusive commits left in the queue; once one side\nis exhausted, no further candidates can appear.\n\n origin/HEAD o o PR HEAD\n | |\n (import) o :\n / \\ /\n | o merge-base\n | |\n : : (~2.5M commits)\n | |\n import root main root\n\n\nIn the RFC thread [1], Derrick Stolee provided a criss-cross counterexample\nthat sharpened the halt condition, and Elijah Newren independently\ndiscovered the same optimization and shared an implementation in PR #2150\n[2]. Patches 2-3 incorporate test cases from Elijah's branch.\n\nThis series implements the optimization only after the walk enters the\nfinite-generation region, where generation ordering guarantees that paint on\nvisited commits is final.\n\nPatch layout:\n\n1/8 Documentation/technical: add paint-down-to-common doc 2/8 t6600: add\ntest cases for side-exhaustion edge cases 3/8 t6099, t6600: add\nside-exhaustion regression tests 4/8 commit-reach: add trace2\ninstrumentation to paint_down_to_common() 5/8 commit-reach: introduce struct\npaint_state with per-side counters 6/8 commit-reach: remove unused\nnonstale_queue dedup wrappers 7/8 commit-reach: terminate merge-base walk\nwhen one paint side is exhausted 8/8 commit-reach: move min_generation check\ninto paint_queue_get()\n\nBenchmarks\n\nStep counts are deterministic (measured via trace2_data_intmax added in\npatch 4). Wall-clock times are best-of-11 runs.\n\n2.6M-commit monorepo with commit-graph:\n\n steps wall-clock\n merge-base --all (across import) 2143438 -> 3 3.67s -> 5ms\n merge-base --all (1000 apart) 2692915 -> 1035 4.41s -> 7ms\n merge-base --all (5000 apart) 2692915 -> 6401 4.45s -> 13ms\n merge-base --all (HEAD vs import) 2698872 -> 45960 4.50s -> 79ms\n merge-tree (across import) 2143438 -> 3 4.42s -> 11ms\n\n\ngit.git (88k commits, commit-graph):\n\n steps wall-clock\n merge-base --all v2.0.0 v2.55.0-rc1 72264 -> 44589 110ms -> 68ms\n merge-base --all HEAD HEAD~1000 9891 -> 3828 18ms -> 10ms\n merge-base --all HEAD HEAD~10000 72303 -> 41487 101ms -> 50ms\n\n\nChanges since v3:\n\n * Fixed BUG assertion that was accidentally made unconditional in v3:\n restored the min_generation guard so it only fires when generation-based\n ordering is active.\n\n * Moved generation cutoff and single-result termination conditions into the\n documentation in patch 1/8, since they describe existing behavior.\n\n * Renamed paint_state counter fields for clarity: p1_count ->\n parent1_count, p2_count -> parent2_count, pending_merge_bases ->\n mb_candidate_count. Changed counter types from int to size_t. (Suggested\n by Rene Scharfe.)\n\nChanges since v2:\n\n * New patch 8/8: moved the min_generation termination check and the\n last_gen monotonicity assertion into paint_queue_get(), consolidating\n halt conditions. commit_graph_generation() is now called once per\n dequeued commit and shared across all checks.\n\n * Moved all halt conditions inside paint_queue_get() with the \"pop first\"\n form: pop, check, then decrement counters. This keeps the optimization\n commit's diff minimal (just inserting the new checks between pop and\n decrement).\n\n * Shortened the doc comment on paint_queue_get() to describe what it does\n rather than how. Inline comments on each return NULL explain the specific\n halt condition.\n\n * Replaced the manual commit-graph setup in the step-count test with\n run_all_modes, which now sets GIT_TRACE2_EVENT per mode and produces\n trace-mode-{none,full,half,no-gdat}.txt files.\n\n * Added a test_paint_down_steps helper for concise 4-mode step assertions\n with diagnostic output on mismatch (prints \"expected X, got Y\" instead of\n a silent grep failure).\n\n * Added step-count assertions to the single-walk edge-case tests:\n in_merge_bases_many:self, pending-stale, infinity-both-sides,\n mixed-finite-infinity.\n\n * Included step counts alongside wall-clock times in the benchmark tables.\n\nChanges since v1:\n\n * Reordered patches: documentation first (describing the existing\n algorithm), tests before code changes, so they demonstrate passing with\n old logic first.\n\n * Dropped the ahead_behind decoupling patch. paint_state is now a NEW\n struct alongside nonstale_queue instead of replacing it. ahead_behind()\n is completely untouched.\n\n * Removed nonstale_queue_put_dedup() and nonstale_queue_get_dedup() (dead\n code after the conversion) in a separate commit.\n\n * Renamed: struct paint_queue -> paint_state, field pq -> queue,\n paint_count_add/remove -> paint_count_update (single function with signed\n delta parameter).\n\n * Split the old paint_count_transition (which handled both old and new\n flags in one call) into separate remove/add calls with a signed delta.\n This eliminates the need for the case 0 handler (which tracked \"not in\n the queue\") and allows an exhaustive switch on (PARENT1 | PARENT2 |\n STALE) that documents all valid flag combinations, with BUG() in default.\n\n * Added trace2_data_intmax() instrumentation to report the number of\n commits visited per paint walk (separate commit), with deterministic\n step-count assertions in t6600.\n\n * Expanded switch statements to multi-line format per .clang-format.\n\n * Used !count style throughout instead of count == 0.\n\n * Updated technical documentation alongside code changes.\n\n[1]\nhttps://lore.kernel.org/git/CAL71e4Ps-2_0+uuZu43N9pFnXBemoAohPs_eyRJf8taXHJPAXQ@mail.gmail.com/T/#u\n[2] https://github.com/gitgitgadget/git/pull/2150\n\nElijah Newren (1):\n t6600: add test cases for side-exhaustion edge cases\n\nKristofer Karlsson (7):\n Documentation/technical: add paint-down-to-common doc\n t6099, t6600: add side-exhaustion regression tests\n commit-reach: add trace2 instrumentation to paint_down_to_common()\n commit-reach: introduce struct paint_state with per-side counters\n commit-reach: remove unused nonstale_queue dedup wrappers\n commit-reach: terminate merge-base walk when one paint side is\n exhausted\n commit-reach: move min_generation check into paint_queue_get()\n\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 149 ++++++++++++++\n commit-reach.c | 147 ++++++++++----\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 ++++++++\n t/t6600-test-reach.sh | 181 ++++++++++++++++--\n 7 files changed, 513 insertions(+), 49 deletions(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\n\nbase-commit: 6c3d7b73556db708feb3b16232fab1efc4353428\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2149%2Fspkrka%2Fside-exhaust-pr-v4\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2149/spkrka/side-exhaust-pr-v4\nPull-Request: https://github.com/gitgitgadget/git/pull/2149\n\nRange-diff vs v3:\n\n 1: 2593866bce ! 1: 3efb095b03 Documentation/technical: add paint-down-to-common doc\n @@ Documentation/technical/paint-down-to-common.adoc (new)\n + 1. The queue is empty.\n + 2. `max_nonstale` has been dequeued, meaning the queue only contains\n + STALE entries.\n ++ 3. Generation cutoff: the dequeued commit's generation is below\n ++ a caller-supplied `min_generation` threshold.\n ++ 4. Single result: the caller only needs one merge base, one has\n ++ been found, and the walk has entered the finite-generation\n ++ region.\n +\n +Stale entry condition\n +~~~~~~~~~~~~~~~~~~~~~\n @@ Documentation/technical/paint-down-to-common.adoc (new)\n +`remove_redundant()` handles that as a post-processing step, so it\n +is safe to exit early.\n +\n ++Generation cutoff\n ++~~~~~~~~~~~~~~~~~\n ++Some callers (notably `remove_redundant()`) supply a `min_generation`\n ++threshold -- the minimum generation of the input commits. No merge\n ++base can have a generation below this threshold, so the walk\n ++terminates as soon as it dequeues such a commit.\n ++\n ++Single result\n ++~~~~~~~~~~~~~\n ++When only one merge base is needed and the walk is in the\n ++finite-generation region, the first candidate found is necessarily\n ++the highest-generation common ancestor. No remaining commit in the\n ++queue can be a descendant of this candidate (generation ordering\n ++guarantees children are visited first), so it cannot be redundant\n ++and the walk can stop immediately.\n ++\n +Related documentation\n +---------------------\n +\n 2: 9efc084850 = 2: 1a0154b406 t6600: add test cases for side-exhaustion edge cases\n 3: 14b0d86b93 = 3: 017bf156c5 t6099, t6600: add side-exhaustion regression tests\n 4: 2592264cda = 4: df3b090a2b commit-reach: add trace2 instrumentation to paint_down_to_common()\n 5: e82e0c72b6 ! 5: fed9f2c368 commit-reach: introduce struct paint_state with per-side counters\n @@ Documentation/technical/paint-down-to-common.adoc: re-enqueued is bounded by the\n - 2. `max_nonstale` has been dequeued, meaning the queue only contains\n - STALE entries.\n + 2. The queue contains only stale entries.\n - \n - Stale entry condition\n - ~~~~~~~~~~~~~~~~~~~~~\n + 3. Generation cutoff: the dequeued commit's generation is below\n + a caller-supplied `min_generation` threshold.\n + 4. Single result: the caller only needs one merge base, one has\n \n ## commit-reach.c ##\n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_q\n + */\n +struct paint_state {\n +\tstruct prio_queue queue;\n -+\tint p1_count;\n -+\tint p2_count;\n -+\tint pending_merge_bases;\n ++\tsize_t parent1_count;\n ++\tsize_t parent2_count;\n ++\tsize_t mb_candidate_count;\n +};\n +\n +static void paint_count_update(struct paint_state *state,\n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_q\n +{\n +\tswitch (flags & (PARENT1 | PARENT2 | STALE)) {\n +\tcase PARENT1:\n -+\t\tstate->p1_count += delta;\n ++\t\tstate->parent1_count += delta;\n +\t\tbreak;\n +\n +\tcase PARENT2:\n -+\t\tstate->p2_count += delta;\n ++\t\tstate->parent2_count += delta;\n +\t\tbreak;\n +\n +\tcase PARENT1 | PARENT2:\n -+\t\tstate->pending_merge_bases += delta;\n ++\t\tstate->mb_candidate_count += delta;\n +\t\tbreak;\n +\n +\tcase PARENT1 | PARENT2 | STALE:\n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_q\n +\n +\tcommit->object.flags &= ~ENQUEUED;\n +\n -+\tif (!state->p1_count && !state->p2_count &&\n -+\t !state->pending_merge_bases)\n ++\tif (!state->parent1_count && !state->parent2_count &&\n ++\t !state->mb_candidate_count)\n +\t\treturn NULL;\n +\n +\tpaint_count_update(state, commit->object.flags, -1);\n 6: e6181bf3c1 = 6: 4db485b48a commit-reach: remove unused nonstale_queue dedup wrappers\n 7: f3572a8a89 ! 7: 4506780649 commit-reach: terminate merge-base walk when one paint side is exhausted\n @@ Commit message\n commit-reach: terminate merge-base walk when one paint side is exhausted\n \n Add an early termination check to paint_down_to_common() using the\n - per-side counters introduced earlier. Once the walk enters the\n + per-side counters introduced earlier. Once the walk enters the\n finite-generation region, terminate early when one side's exclusive\n count drops to zero -- no new merge-base can form without both paint\n sides meeting.\n @@ Commit message\n \n The INFINITY gate ensures correctness: commits without a commit-graph\n entry have GENERATION_NUMBER_INFINITY and are ordered by commit date,\n - which is not topologically reliable. The optimization only fires\n + which is not topologically reliable. The optimization only fires\n once the walk enters the finite-generation region where ordering\n guarantees hold.\n \n - Widen the existing generation-monotonicity BUG assertion to fire\n - unconditionally, not only when min_generation is set. The\n - side-exhaustion optimization depends on correct generation ordering,\n - so the assertion should always be active.\n -\n Step counts measured with trace2 on git.git with commit-graph:\n \n merge-base --all v2.0.0 v2.55.0-rc1:\n @@ Commit message\n \n ## Documentation/technical/paint-down-to-common.adoc ##\n @@ Documentation/technical/paint-down-to-common.adoc: ends when one of the following conditions holds:\n - \n - 1. The queue is empty.\n - 2. The queue contains only stale entries.\n -+ 3. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n + 4. Single result: the caller only needs one merge base, one has\n + been found, and the walk has entered the finite-generation\n + region.\n ++ 5. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n + remain in the queue, no pending merge-base candidates exist,\n + and the walk has entered the finite-generation region.\n \n @@ Documentation/technical/paint-down-to-common.adoc: existing candidates by provin\n +commit-date ordering can violate this guarantee, so the check is\n +skipped.\n +\n - Related documentation\n - ---------------------\n - \n + Generation cutoff\n + ~~~~~~~~~~~~~~~~~\n + Some callers (notably `remove_redundant()`) supply a `min_generation`\n \n ## commit-reach.c ##\n @@ commit-reach.c: static void paint_queue_put(struct paint_state *state,\n @@ commit-reach.c: static struct commit *paint_queue_get(struct paint_state *state)\n \n \tcommit->object.flags &= ~ENQUEUED;\n \n --\tif (!state->p1_count && !state->p2_count &&\n --\t !state->pending_merge_bases)\n +-\tif (!state->parent1_count && !state->parent2_count &&\n +-\t !state->mb_candidate_count)\n -\t\treturn NULL;\n -+\tif (!state->pending_merge_bases) {\n ++\tif (!state->mb_candidate_count) {\n +\t\t/* only stale entries remain */\n -+\t\tif (!state->p1_count && !state->p2_count)\n ++\t\tif (!state->parent1_count && !state->parent2_count)\n +\t\t\treturn NULL;\n +\n +\t\t/* one side is exhausted */\n -+\t\tif ((!state->p1_count || !state->p2_count) &&\n ++\t\tif ((!state->parent1_count || !state->parent2_count) &&\n +\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n +\t\t\treturn NULL;\n +\t}\n \n \tpaint_count_update(state, commit->object.flags, -1);\n \treturn commit;\n -@@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n - \t\ttimestamp_t generation = commit_graph_generation(commit);\n - \t\tsteps++;\n - \n --\t\tif (min_generation && generation > last_gen)\n -+\t\tif (generation > last_gen)\n - \t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n - \t\t\t generation, last_gen,\n - \t\t\t oid_to_hex(&commit->object.oid));\n \n ## t/t6600-test-reach.sh ##\n @@ t/t6600-test-reach.sh: test_expect_success 'in_merge_bases_many:self' '\n 8: 4b9f192d98 ! 8: 8dd15d44e6 commit-reach: move min_generation check into paint_queue_get()\n @@ Commit message\n Move last_gen into struct paint_state so that\n commit_graph_generation() is called exactly once per dequeued commit\n and the result is shared across all termination checks and the\n - monotonicity BUG assertion. The loop body in paint_down_to_common()\n - reads state.last_gen instead of recomputing the generation number.\n + monotonicity BUG assertion.\n \n Signed-off-by: Kristofer Karlsson \n \n - ## Documentation/technical/paint-down-to-common.adoc ##\n -@@ Documentation/technical/paint-down-to-common.adoc: ends when one of the following conditions holds:\n - 3. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n - remain in the queue, no pending merge-base candidates exist,\n - and the walk has entered the finite-generation region.\n -+ 4. Generation cutoff: the dequeued commit's generation is below\n -+ a caller-supplied `min_generation` threshold.\n - \n - Stale entry condition\n - ~~~~~~~~~~~~~~~~~~~~~\n -@@ Documentation/technical/paint-down-to-common.adoc: time and an exhausted side cannot reappear. In the INFINITY region,\n - commit-date ordering can violate this guarantee, so the check is\n - skipped.\n - \n -+Generation cutoff\n -+~~~~~~~~~~~~~~~~~\n -+Some callers (notably `remove_redundant()`) supply a `min_generation`\n -+threshold -- the minimum generation of the input commits. No merge\n -+base can have a generation below this threshold, so the walk\n -+terminates as soon as it dequeues such a commit.\n -+\n - Related documentation\n - ---------------------\n - \n -\n ## commit-reach.c ##\n @@ commit-reach.c: struct paint_state {\n - \tint p1_count;\n - \tint p2_count;\n - \tint pending_merge_bases;\n + \tsize_t parent1_count;\n + \tsize_t parent2_count;\n + \tsize_t mb_candidate_count;\n +\ttimestamp_t min_generation;\n +\ttimestamp_t last_gen;\n };\n @@ commit-reach.c: static void paint_queue_put(struct paint_state *state,\n \tcommit->object.flags &= ~ENQUEUED;\n +\tgeneration = commit_graph_generation(commit);\n +\n -+\tif (generation > state->last_gen)\n ++\tif (state->min_generation && generation > state->last_gen)\n +\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n +\t\t generation, state->last_gen,\n +\t\t oid_to_hex(&commit->object.oid));\n @@ commit-reach.c: static void paint_queue_put(struct paint_state *state,\n +\tif (generation < state->min_generation)\n +\t\treturn NULL;\n \n - \tif (!state->pending_merge_bases) {\n + \tif (!state->mb_candidate_count) {\n \t\t/* only stale entries remain */\n @@ commit-reach.c: static struct commit *paint_queue_get(struct paint_state *state)\n \n \t\t/* one side is exhausted */\n - \t\tif ((!state->p1_count || !state->p2_count) &&\n + \t\tif ((!state->parent1_count || !state->parent2_count) &&\n -\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n +\t\t generation < GENERATION_NUMBER_INFINITY)\n \t\t\treturn NULL;\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n -\t\ttimestamp_t generation = commit_graph_generation(commit);\n \t\tsteps++;\n \n --\t\tif (generation > last_gen)\n +-\t\tif (min_generation && generation > last_gen)\n -\t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n -\t\t\t generation, last_gen,\n -\t\t\t oid_to_hex(&commit->object.oid));\n\n-- \ngitgitgadget\n"},{"id":"546601","messageId":"3efb095b0345a16f3478003d88a0b633830b2dc1.1782649547.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v4 1/8] Documentation/technical: add paint-down-to-common doc","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:39Z","receivedAt":"2026-06-28T12:25:54Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a technical document describing the paint_down_to_common()\nalgorithm used for merge-base computation, covering the paint\nwalk, generation number regions, and termination conditions.\n\nSigned-off-by: Kristofer Karlsson \n---\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 135 ++++++++++++++++++\n commit-reach.c | 6 +-\n 4 files changed, 142 insertions(+), 1 deletion(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 2699f0b24a..f8dea4b395 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -129,6 +129,7 @@ TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\n TECH_DOCS += technical/packfile-uri\n TECH_DOCS += technical/pack-heuristics\n+TECH_DOCS += technical/paint-down-to-common\n TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/platform-support\ndiff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\nindex ec07088c57..9ce11d5e48 100644\n--- a/Documentation/technical/meson.build\n+++ b/Documentation/technical/meson.build\n@@ -18,6 +18,7 @@ articles = [\n 'multi-pack-index.adoc',\n 'packfile-uri.adoc',\n 'pack-heuristics.adoc',\n+ 'paint-down-to-common.adoc',\n 'parallel-checkout.adoc',\n 'partial-clone.adoc',\n 'platform-support.adoc',\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nnew file mode 100644\nindex 0000000000..a4dfcba038\n--- /dev/null\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -0,0 +1,135 @@\n+Merge-Base Computation and paint_down_to_common()\n+==================================================\n+\n+The function `paint_down_to_common()` in `commit-reach.c` computes merge\n+bases by walking the commit graph backwards from two sets of tips and\n+finding where their ancestry meets.\n+\n+Use cases\n+---------\n+\n+Computing merge bases is used in two different ways:\n+\n+ 1. *Finding all merge bases* (`merge-base --all`, `merge-tree`,\n+ `merge`, `rebase`). A merge base is a common ancestor that is\n+ not itself an ancestor of another common ancestor.\n+\n+ 2. *Ancestry checks* (`in_merge_bases`, used by `merge-base\n+ --is-ancestor`, `branch -d`, `fetch`). These ask: \"is commit A\n+ an ancestor of commit B?\" If a common ancestor equals one of the\n+ inputs, that input is necessarily the only merge base -- no other\n+ common ancestor can be both as recent and not an ancestor of it.\n+\n+Both use cases share the same algorithm and implementation.\n+\n+Algorithm\n+---------\n+\n+Given a commit `one` and a set of commits `twos[]`, the walk paints\n+commits with two colors:\n+\n+ - PARENT1: reachable from `one`\n+ - PARENT2: reachable from any commit in `twos[]`\n+\n+The walk uses a priority queue ordered by generation number (falling\n+back to commit date when generation numbers are unavailable). Each\n+step dequeues the highest-priority commit (this is when we say a\n+commit is \"visited\") and propagates its paint flags to its parents,\n+enqueuing them if they gained new flags. When a commit receives\n+both PARENT1 and PARENT2, it is a merge-base candidate. A candidate\n+gains the STALE flag so its ancestors propagate staleness -- any\n+deeper common ancestor is necessarily redundant.\n+\n+INFINITY and finite generation regions\n+--------------------------------------\n+\n+The commit-graph stores a generation number for each commit. Commits\n+not in the commit-graph have generation `GENERATION_NUMBER_INFINITY`. The\n+graph is closed under reachability: if a commit is in the graph, all\n+its ancestors are too. This partitions the commit graph into two regions:\n+\n+....\n+ +---------------------------------------+\n+ | INFINITY region |\n+ | generation = INFINITY |\n+ | queue order: heuristic (commit date) |\n+ +---------------------------------------+\n+ |\n+ v\n+ +---------------------------------------+\n+ | Finite region |\n+ | generation = finite |\n+ | queue order: topological |\n+ +---------------------------------------+\n+....\n+\n+When the commit-graph is enabled, the INFINITY region is typically\n+very small -- it only contains commits added since the last\n+commit-graph refresh.\n+\n+All reachable INFINITY-generation commits are visited before any\n+finite-generation commit, because INFINITY is larger than any finite\n+value. Once the walk crosses into the finite region, it stays there.\n+\n+In the finite region, generation ordering guarantees topological\n+traversal: children are always visited before their parents. This\n+means that paint on already-visited commits is final -- no future\n+traversal step can add paint to them.\n+\n+In the INFINITY region, commit-date ordering can violate this: a\n+parent with a later date can be visited before a child with an earlier\n+date. Paint flags are therefore NOT final at visit time, and a\n+commit visited with only one side's paint may later gain the other.\n+\n+Paint flags are only added, never removed. Since each flag can be set\n+at most once per commit, the number of times a commit can be\n+re-enqueued is bounded by the number of flag transitions.\n+\n+Termination\n+-----------\n+\n+The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n+tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n+so far. Once that commit is dequeued, every remaining entry is known\n+to be STALE and the loop terminates. Specifically, the main loop\n+ends when one of the following conditions holds:\n+\n+ 1. The queue is empty.\n+ 2. `max_nonstale` has been dequeued, meaning the queue only contains\n+ STALE entries.\n+ 3. Generation cutoff: the dequeued commit's generation is below\n+ a caller-supplied `min_generation` threshold.\n+ 4. Single result: the caller only needs one merge base, one has\n+ been found, and the walk has entered the finite-generation\n+ region.\n+\n+Stale entry condition\n+~~~~~~~~~~~~~~~~~~~~~\n+Once all queued entries are stale, no new merge-base candidates can\n+be discovered -- that requires at least one non-stale commit from\n+each side meeting. Continuing the walk could still invalidate\n+existing candidates by proving one is an ancestor of another, but\n+`remove_redundant()` handles that as a post-processing step, so it\n+is safe to exit early.\n+\n+Generation cutoff\n+~~~~~~~~~~~~~~~~~\n+Some callers (notably `remove_redundant()`) supply a `min_generation`\n+threshold -- the minimum generation of the input commits. No merge\n+base can have a generation below this threshold, so the walk\n+terminates as soon as it dequeues such a commit.\n+\n+Single result\n+~~~~~~~~~~~~~\n+When only one merge base is needed and the walk is in the\n+finite-generation region, the first candidate found is necessarily\n+the highest-generation common ancestor. No remaining commit in the\n+queue can be a descendant of this candidate (generation ordering\n+guarantees children are visited first), so it cannot be redundant\n+and the walk can stop immediately.\n+\n+Related documentation\n+---------------------\n+\n+ - `Documentation/technical/commit-graph.adoc` -- generation numbers\n+ and the reachability closure property.\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 5df471a313..a9483759e0 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -96,7 +96,11 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n \treturn commit;\n }\n \n-/* all input commits in one and twos[] must have been parsed! */\n+/*\n+ * See Documentation/technical/paint-down-to-common.adoc\n+ *\n+ * All input commits in one and twos[] must have been parsed!\n+ */\n static int paint_down_to_common(struct repository *r,\n \t\t\t\tstruct commit *one, int n,\n \t\t\t\tstruct commit **twos,\n-- \ngitgitgadget\n\n"},{"id":"546602","messageId":"1a0154b406343ba7e43bc60da45c4a5deadc7e30.1782649547.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v4 2/8] t6600: add test cases for side-exhaustion edge cases","fromName":"Elijah Newren via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:40Z","receivedAt":"2026-06-28T12:25:56Z","isPatch":true,"body":"From: Elijah Newren \n\nAdd test cases to t6600-test-reach.sh that exercise edge cases in the\nside-exhaustion optimization for paint_down_to_common():\n\n - in_merge_bases_many:self: commit is both A and one of the X inputs\n - get_merge_bases_many:duplicate-twos: duplicate entries in X list\n - get_merge_bases_many:pending-stale: STALE transition on an\n already-painted commit (ps-* diamond topology)\n - get_merge_bases_many:infinity-both-sides: both tips outside the\n commit-graph with non-monotonic dates (pi-* topology)\n\nSigned-off-by: Elijah Newren \nSigned-off-by: Kristofer Karlsson \n---\n t/t6600-test-reach.sh | 111 ++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 111 insertions(+)\n\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex b5b314e570..c2e091aad1 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -49,6 +49,62 @@ test_expect_success 'setup' '\n \t\t\tgit tag -a -m \"$x-$i\" tag-$x-$i commit-$x-$i || return 1\n \t\tdone\n \tdone &&\n+\n+\t# Build a small side topology to exercise the (PARENT1|PARENT2) ->\n+\t# (PARENT1|PARENT2|STALE) transition in paint_down_to_common(); the\n+\t# 10x10 grid above does not exercise it because no merge-base candidate\n+\t# there is a descendant of another, so STALE never reaches a\n+\t# still-pending candidate.\n+\t#\n+\t# ps-X\n+\t# /|\\\n+\t# / | \\\n+\t# ps-Z ps-B ps-W\n+\t# | / \\ |\n+\t# | / \\ |\n+\t# |/ \\|\n+\t# ps-T1 ps-T2\n+\t#\n+\t# where ps-T1=merge(ps-Z,ps-B), ps-T2=merge(ps-W,ps-B), so\n+\t# merge-base(ps-T1,ps-T2) = ps-B. During the walk, ps-X transitions\n+\t# to (PARENT1|PARENT2) via ps-Z and ps-W before ps-B is dequeued;\n+\t# then the STALE-walk from ps-B transitions ps-X to\n+\t# (PARENT1|PARENT2|STALE).\n+\tgit checkout --orphan ps-orphan &&\n+\ttest_commit ps-X &&\n+\tgit checkout -b ps-B-br ps-X && test_commit ps-B &&\n+\tgit checkout -b ps-Z-br ps-X && test_commit ps-Z &&\n+\tgit checkout -b ps-W-br ps-X && test_commit ps-W &&\n+\tgit checkout -b ps-T1 ps-Z &&\n+\tgit merge --no-ff -m ps-T1 ps-B &&\n+\tgit checkout -b ps-T2 ps-W &&\n+\tgit merge --no-ff -m ps-T2 ps-B &&\n+\n+\t# Build a side topology that lives entirely outside the half\n+\t# commit-graph and has non-monotonic commit dates, to exercise the\n+\t# INFINITY-gate in paint_down_to_common. With both tips outside\n+\t# the graph, generation is INFINITY and the queue falls back to\n+\t# commit-date order, which here is non-monotonic.\n+\t#\n+\t# pi-X (date 500, PARENT1 tip) --> pi-P, pi-D\n+\t# pi-D (date 480) --> pi-C\n+\t# pi-C (date 200) --> pi-B\n+\t# pi-B (date 100, PARENT2 tip) --> pi-P\n+\t# pi-P (date 450, root)\n+\t#\n+\t# merge-base(pi-X, pi-B) = pi-B (it is an ancestor of pi-X and is\n+\t# itself one of the queried tips).\n+\tgit checkout --orphan pi-orphan &&\n+\ttest_commit --date \"@450 +0000\" pi-P &&\n+\ttest_commit --date \"@100 +0000\" pi-B &&\n+\ttest_commit --date \"@200 +0000\" pi-C &&\n+\ttest_commit --date \"@480 +0000\" pi-D &&\n+\tGIT_AUTHOR_DATE=\"@500 +0000\" GIT_COMMITTER_DATE=\"@500 +0000\" \\\n+\t\tgit commit-tree -p pi-D -p pi-P -m pi-X pi-D^{tree} >pi-X-oid &&\n+\tpi_x=\"$(cat pi-X-oid)\" &&\n+\tgit branch -f pi-X-br \"$pi_x\" &&\n+\tgit tag pi-X \"$pi_x\" &&\n+\n \tgit commit-graph write --reachable &&\n \tmv .git/objects/info/commit-graph commit-graph-full &&\n \tchmod u+w commit-graph-full &&\n@@ -146,6 +202,16 @@ test_expect_success 'in_merge_bases_many:miss-heuristic' '\n \ttest_all_modes in_merge_bases_many\n '\n \n+test_expect_success 'in_merge_bases_many:self' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-6-8\n+\tX:commit-5-9\n+\tX:commit-6-8\n+\tEOF\n+\techo \"in_merge_bases_many(A,X):1\" >expect &&\n+\ttest_all_modes in_merge_bases_many\n+'\n+\n test_expect_success 'is_descendant_of:hit' '\n \tcat >input <<-\\EOF &&\n \tA:commit-5-7\n@@ -183,6 +249,51 @@ test_expect_success 'get_merge_bases_many' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'get_merge_bases_many:duplicate-twos' '\n+\tcat >input <<-\\EOF &&\n+\tA:commit-5-7\n+\tX:commit-4-8\n+\tX:commit-4-8\n+\tX:commit-6-6\n+\tX:commit-6-6\n+\tX:commit-8-3\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse commit-5-6 \\\n+\t\t\t commit-4-7 | sort\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n+test_expect_success 'get_merge_bases_many:pending-stale' '\n+\t# Exercises the (PARENT1|PARENT2) -> (...|STALE) transition path in\n+\t# paint_down_to_common(). See the topology comment in the setup test.\n+\tcat >input <<-\\EOF &&\n+\tA:ps-T1\n+\tX:ps-T2\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse ps-B\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n+test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n+\t# Exercises the push-time INFINITY-gate in paint_down_to_common(). See\n+\t# the pi-* topology comment in the setup test.\n+\tcat >input <<-\\EOF &&\n+\tA:pi-X\n+\tX:pi-B\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse pi-B\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546603","messageId":"017bf156c57ceb2df33b316d4537baf3a72672fd.1782649547.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v4 3/8] t6099, t6600: add side-exhaustion regression tests","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:41Z","receivedAt":"2026-06-28T12:25:57Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd t6099 to test the case where multiple merge-base candidates exist\nand one is an ancestor of another. This exercises the side-exhaustion\noptimization in paint_down_to_common together with the\nremove_redundant safety net in get_merge_bases_many_0.\n\nAdd a mixed finite/INFINITY test to t6600 where one tip is outside\nthe commit-graph (INFINITY generation) and the other is inside.\nThis exercises the region transition: the walk starts in the\nINFINITY region where side-exhaustion is disabled, then crosses\ninto the finite region where it can fire.\n\nSigned-off-by: Kristofer Karlsson \n---\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 +++++++++++++++++++++++++++\n t/t6600-test-reach.sh | 25 ++++++++\n 3 files changed, 108 insertions(+)\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\ndiff --git a/t/meson.build b/t/meson.build\nindex 3219264fe7..ee6ebdffb9 100644\n--- a/t/meson.build\n+++ b/t/meson.build\n@@ -786,6 +786,7 @@ integration_tests = [\n 't6041-bisect-submodule.sh',\n 't6050-replace.sh',\n 't6060-merge-index.sh',\n+ 't6099-merge-base-side-exhaustion.sh',\n 't6100-rev-list-in-order.sh',\n 't6101-rev-parse-parents.sh',\n 't6102-rev-list-unexpected-objects.sh',\ndiff --git a/t/t6099-merge-base-side-exhaustion.sh b/t/t6099-merge-base-side-exhaustion.sh\nnew file mode 100755\nindex 0000000000..4f1e0d50ef\n--- /dev/null\n+++ b/t/t6099-merge-base-side-exhaustion.sh\n@@ -0,0 +1,82 @@\n+#!/bin/sh\n+\n+test_description='merge-base with ancestor among merge-base candidates\n+\n+Test that merge-base --all correctly handles cases where\n+multiple merge-base candidates exist and one is an ancestor\n+of another. The side-exhaustion optimization in\n+paint_down_to_common may exit before STALE propagation\n+removes the ancestor, but remove_redundant catches it.\n+\n+Graph shape (parents are below children):\n+\n+ A ----------- X\n+ |\\ /|\n+ | B---------/ |\n+ | | |\n+ e2 \\ f2\n+ | | |\n+ e1 d1 f1\n+ \\ | /\n+ \\ | /\n+ \\| /\n+ C\n+\n+A and X are the two tips.\n+B and C are both reachable from A and X.\n+B reaches C through d1.\n+Only B should appear in merge-base --all output.\n+'\n+\n+GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main\n+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME\n+\n+TEST_PASSES_SANITIZE_LEAK=true\n+. ./test-lib.sh\n+\n+test_expect_success 'setup ancestor merge-base candidate' '\n+\ttest_commit C &&\n+\n+\tgit checkout -b d-chain HEAD &&\n+\ttest_commit d1 &&\n+\ttest_commit B &&\n+\n+\tgit checkout -b e-path C &&\n+\ttest_commit e1 &&\n+\ttest_commit e2 &&\n+\n+\tgit checkout -b f-path C &&\n+\ttest_commit f1 &&\n+\ttest_commit f2 &&\n+\n+\tgit checkout -b branch-A e-path &&\n+\ttest_merge A B &&\n+\n+\tgit checkout -b branch-X f-path &&\n+\ttest_merge X B &&\n+\n+\tgit commit-graph write --reachable\n+'\n+\n+test_expect_success 'merge-base --all excludes ancestor candidate' '\n+\tgit rev-parse B >expected &&\n+\tgit merge-base --all A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_expect_success 'merge-base (single) finds shallowest' '\n+\tgit rev-parse B >expected &&\n+\tgit merge-base A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+# Without commit-graph: generation numbers are INFINITY,\n+# side-exhaustion optimization does not fire.\n+test_expect_success 'merge-base --all without commit-graph' '\n+\trm -f .git/objects/info/commit-graph &&\n+\tgit rev-parse B >expected &&\n+\tgit merge-base --all A X >actual &&\n+\ttest_cmp expected actual\n+'\n+\n+test_done\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex c2e091aad1..4b771b4c58 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -294,6 +294,31 @@ test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n \ttest_all_modes get_merge_bases_many\n '\n \n+test_expect_success 'setup mixed finite/INFINITY topology' '\n+\t# Create a commit outside all saved commit-graph files so it always\n+\t# has INFINITY generation, while its parent (ps-X) is in the graph\n+\t# with a finite generation. Use the ps-* orphan topology so we do\n+\t# not pollute the grid-based rev-list tests.\n+\tgit checkout ps-X &&\n+\ttest_env GIT_TEST_COMMIT_GRAPH= test_commit pm-INF\n+'\n+\n+test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n+\t# One tip (pm-INF) is outside the commit-graph with INFINITY\n+\t# generation; the other (ps-B) is in the graph with finite\n+\t# generation. The walk starts in the INFINITY region and crosses\n+\t# into the finite region where side-exhaustion can fire.\n+\tcat >input <<-\\EOF &&\n+\tA:pm-INF\n+\tX:ps-B\n+\tEOF\n+\t{\n+\t\techo \"get_merge_bases_many(A,X):\" &&\n+\t\tgit rev-parse ps-X\n+\t} >expect &&\n+\ttest_all_modes get_merge_bases_many\n+'\n+\n test_expect_success 'reduce_heads' '\n \tcat >input <<-\\EOF &&\n \tX:commit-1-10\n-- \ngitgitgadget\n\n"},{"id":"546604","messageId":"df3b090a2bf43737cb9b6ac9b4e769a658250bb5.1782649547.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v4 4/8] commit-reach: add trace2 instrumentation to paint_down_to_common()","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:42Z","receivedAt":"2026-06-28T12:26:00Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a step counter and trace2_data_intmax() call so that the number\nof commits visited during the paint walk is observable via\nGIT_TRACE2_EVENT. This provides a way to measure the impact of\nfuture optimizations without relying on wall-clock benchmarks alone.\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 5 ++++\n t/t6600-test-reach.sh | 53 ++++++++++++++++++++++++++++++-------------\n 2 files changed, 42 insertions(+), 16 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex a9483759e0..f6a438550b 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -11,6 +11,7 @@\n #include \"tag.h\"\n #include \"commit-reach.h\"\n #include \"ewah/ewok.h\"\n+#include \"trace2.h\"\n \n /* Remember to update object flag allocation in object.h */\n #define PARENT1\t\t(1u<<16)\n@@ -112,6 +113,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t{ compare_commits_by_gen_then_commit_date }\n \t};\n \tint i;\n+\tint steps = 0;\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n@@ -135,6 +137,7 @@ static int paint_down_to_common(struct repository *r,\n \t\tstruct commit_list *parents;\n \t\tint flags;\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n+\t\tsteps++;\n \n \t\tif (min_generation && generation > last_gen)\n \t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n@@ -190,6 +193,8 @@ static int paint_down_to_common(struct repository *r,\n \t}\n \n \tclear_nonstale_queue(&queue);\n+\ttrace2_data_intmax(\"paint_down_to_common\", r,\n+\t\t\t \"steps\", steps);\n \tcommit_list_sort_by_date(result);\n \treturn 0;\n }\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 4b771b4c58..b3a31b80ac 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -118,24 +118,34 @@ test_expect_success 'setup' '\n '\n \n run_all_modes () {\n-\ttest_when_finished rm -rf .git/objects/info/commit-graph &&\n-\t\"$@\" actual &&\n-\ttest_cmp expect actual &&\n-\tcp commit-graph-full .git/objects/info/commit-graph &&\n-\t\"$@\" actual &&\n-\ttest_cmp expect actual &&\n-\tcp commit-graph-half .git/objects/info/commit-graph &&\n-\t\"$@\" actual &&\n-\ttest_cmp expect actual &&\n-\tcp commit-graph-no-gdat .git/objects/info/commit-graph &&\n-\t\"$@\" actual &&\n-\ttest_cmp expect actual\n+\tgraph=.git/objects/info/commit-graph &&\n+\ttest_when_finished rm -rf \"$graph\" \"${graph}s\" &&\n+\trm -f trace-mode-*.txt &&\n+\n+\tfor mode in none full half no-gdat\n+\tdo\n+\t\trm -rf \"$graph\" \"${graph}s\" &&\n+\t\tcp \"commit-graph-${mode}\" \"$graph\" 2>/dev/null ||\n+\t\ttrue &&\n+\t\tGIT_TRACE2_EVENT=\"$(pwd)/trace-mode-${mode}.txt\" \\\n+\t\t\t\"$@\" actual &&\n+\t\ttest_cmp expect actual || return 1\n+\tdone\n }\n \n test_all_modes () {\n \trun_all_modes test-tool reach \"$@\"\n }\n \n+test_paint_down_steps () {\n+\tfor mode in none full half no-gdat\n+\tdo\n+\t\ttest_trace2_data paint_down_to_common steps \"$1\" \\\n+\t\t\t<\"trace-mode-${mode}.txt\" || return 1\n+\t\tshift\n+\tdone\n+}\n+\n test_expect_success 'ref_newer:miss' '\n \tcat >input <<-\\EOF &&\n \tA:commit-5-7\n@@ -209,7 +219,8 @@ test_expect_success 'in_merge_bases_many:self' '\n \tX:commit-6-8\n \tEOF\n \techo \"in_merge_bases_many(A,X):1\" >expect &&\n-\ttest_all_modes in_merge_bases_many\n+\ttest_all_modes in_merge_bases_many &&\n+\ttest_paint_down_steps 45 2 25 3\n '\n \n test_expect_success 'is_descendant_of:hit' '\n@@ -277,7 +288,8 @@ test_expect_success 'get_merge_bases_many:pending-stale' '\n \t\techo \"get_merge_bases_many(A,X):\" &&\n \t\tgit rev-parse ps-B\n \t} >expect &&\n-\ttest_all_modes get_merge_bases_many\n+\ttest_all_modes get_merge_bases_many &&\n+\ttest_paint_down_steps 6 6 6 6\n '\n \n test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n@@ -291,7 +303,8 @@ test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n \t\techo \"get_merge_bases_many(A,X):\" &&\n \t\tgit rev-parse pi-B\n \t} >expect &&\n-\ttest_all_modes get_merge_bases_many\n+\ttest_all_modes get_merge_bases_many &&\n+\ttest_paint_down_steps 5 5 5 5\n '\n \n test_expect_success 'setup mixed finite/INFINITY topology' '\n@@ -316,7 +329,15 @@ test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n \t\techo \"get_merge_bases_many(A,X):\" &&\n \t\tgit rev-parse ps-X\n \t} >expect &&\n-\ttest_all_modes get_merge_bases_many\n+\ttest_all_modes get_merge_bases_many &&\n+\ttest_paint_down_steps 3 3 3 3\n+'\n+\n+test_expect_success 'merge-base --all commit-walk steps' '\n+\t>input &&\n+\tgit rev-parse commit-9-1 >expect &&\n+\trun_all_modes git merge-base --all commit-9-9 commit-9-1 &&\n+\ttest_paint_down_steps 81 80 81 81\n '\n \n test_expect_success 'reduce_heads' '\n-- \ngitgitgadget\n\n"},{"id":"546605","messageId":"fed9f2c36827f17d636e0ef986b358c15fd86239.1782649547.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v4 5/8] commit-reach: introduce struct paint_state with per-side counters","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:43Z","receivedAt":"2026-06-28T12:26:03Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a paint_state struct for use by paint_down_to_common() that\nwraps a prio_queue with per-side commit counters. Each non-stale\nqueued commit occupies exactly one counter bucket based on its\npaint flags: PARENT1-only, PARENT2-only, or both sides (a pending\nmerge-base candidate).\n\nThe counters are maintained by paint_count_update() which adjusts\nthe appropriate bucket by a signed delta. An exhaustive switch on\nthe paint+stale bits documents all valid flag combinations in one\nplace.\n\nConvert paint_down_to_common() to use paint_state. The loop now\ndrains the queue via paint_queue_get() which returns NULL when all\ncounters reach zero, replacing the old pointer-based termination\n(max_nonstale). This is equivalent behavior -- both conditions\ndetect that no non-stale entries remain.\n\npaint_queue_get() uses a \"pop first\" form: it dequeues a commit,\nthen checks the counters. This means the loop exits one iteration\nearlier than the old code in some topologies (the popped stale\ncommit is never processed), so a few step counts drop by one.\n\nThe existing nonstale_queue is left in place for ahead_behind().\n\nSigned-off-by: Kristofer Karlsson \n---\n .../technical/paint-down-to-common.adoc | 9 +-\n commit-reach.c | 94 ++++++++++++++++---\n t/t6600-test-reach.sh | 4 +-\n 3 files changed, 85 insertions(+), 22 deletions(-)\n\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nindex a4dfcba038..ac3e2b39a5 100644\n--- a/Documentation/technical/paint-down-to-common.adoc\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -88,15 +88,12 @@ re-enqueued is bounded by the number of flag transitions.\n Termination\n -----------\n \n-The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n-tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n-so far. Once that commit is dequeued, every remaining entry is known\n-to be STALE and the loop terminates. Specifically, the main loop\n+The walk tracks the number of commits of each type in the queue\n+(PARENT1-only, PARENT2-only, pending merge-base). The main loop\n ends when one of the following conditions holds:\n \n 1. The queue is empty.\n- 2. `max_nonstale` has been dequeued, meaning the queue only contains\n- STALE entries.\n+ 2. The queue contains only stale entries.\n 3. Generation cutoff: the dequeued commit's generation is below\n a caller-supplied `min_generation` threshold.\n 4. Single result: the caller only needs one merge base, one has\ndiff --git a/commit-reach.c b/commit-reach.c\nindex f6a438550b..9ae306f60c 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -97,6 +97,75 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n \treturn commit;\n }\n \n+/*\n+ * Priority queue with per-side commit counters for paint_down_to_common().\n+ * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n+ * PARENT2-only, or both (a pending merge-base candidate).\n+ */\n+struct paint_state {\n+\tstruct prio_queue queue;\n+\tsize_t parent1_count;\n+\tsize_t parent2_count;\n+\tsize_t mb_candidate_count;\n+};\n+\n+static void paint_count_update(struct paint_state *state,\n+\t\t\t unsigned flags, int delta)\n+{\n+\tswitch (flags & (PARENT1 | PARENT2 | STALE)) {\n+\tcase PARENT1:\n+\t\tstate->parent1_count += delta;\n+\t\tbreak;\n+\n+\tcase PARENT2:\n+\t\tstate->parent2_count += delta;\n+\t\tbreak;\n+\n+\tcase PARENT1 | PARENT2:\n+\t\tstate->mb_candidate_count += delta;\n+\t\tbreak;\n+\n+\tcase PARENT1 | PARENT2 | STALE:\n+\t\tbreak;\n+\n+\tdefault:\n+\t\tBUG(\"unexpected paint state\");\n+\t}\n+}\n+\n+static void paint_queue_put(struct paint_state *state,\n+\t\t\t struct commit *c, unsigned add_flags)\n+{\n+\tunsigned old_flags = c->object.flags;\n+\tc->object.flags |= add_flags;\n+\n+\tif (old_flags & ENQUEUED) {\n+\t\tpaint_count_update(state, old_flags, -1);\n+\t\tpaint_count_update(state, c->object.flags, 1);\n+\t} else {\n+\t\tc->object.flags |= ENQUEUED;\n+\t\tprio_queue_put(&state->queue, c);\n+\t\tpaint_count_update(state, c->object.flags, 1);\n+\t}\n+}\n+\n+static struct commit *paint_queue_get(struct paint_state *state)\n+{\n+\tstruct commit *commit = prio_queue_get(&state->queue);\n+\n+\tif (!commit)\n+\t\treturn NULL;\n+\n+\tcommit->object.flags &= ~ENQUEUED;\n+\n+\tif (!state->parent1_count && !state->parent2_count &&\n+\t !state->mb_candidate_count)\n+\t\treturn NULL;\n+\n+\tpaint_count_update(state, commit->object.flags, -1);\n+\treturn commit;\n+}\n+\n /*\n * See Documentation/technical/paint-down-to-common.adoc\n *\n@@ -109,31 +178,29 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\tenum merge_base_flags mb_flags,\n \t\t\t\tstruct commit_list **result)\n {\n-\tstruct nonstale_queue queue = {\n-\t\t{ compare_commits_by_gen_then_commit_date }\n+\tstruct paint_state state = {\n+\t\t.queue = { compare_commits_by_gen_then_commit_date }\n \t};\n+\tstruct commit *commit;\n \tint i;\n \tint steps = 0;\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n \tif (!min_generation && !corrected_commit_dates_enabled(r))\n-\t\tqueue.pq.compare = compare_commits_by_commit_date;\n+\t\tstate.queue.compare = compare_commits_by_commit_date;\n \n \tone->object.flags |= PARENT1;\n \tif (!n) {\n \t\tcommit_list_append(one, result);\n \t\treturn 0;\n \t}\n-\tnonstale_queue_put_dedup(&queue, one);\n+\tpaint_queue_put(&state, one, 0);\n \n-\tfor (i = 0; i < n; i++) {\n-\t\ttwos[i]->object.flags |= PARENT2;\n-\t\tnonstale_queue_put_dedup(&queue, twos[i]);\n-\t}\n+\tfor (i = 0; i < n; i++)\n+\t\tpaint_queue_put(&state, twos[i], PARENT2);\n \n-\twhile (queue.max_nonstale) {\n-\t\tstruct commit *commit = nonstale_queue_get_dedup(&queue);\n+\twhile ((commit = paint_queue_get(&state))) {\n \t\tstruct commit_list *parents;\n \t\tint flags;\n \t\ttimestamp_t generation = commit_graph_generation(commit);\n@@ -172,7 +239,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t\tif ((p->object.flags & flags) == flags)\n \t\t\t\tcontinue;\n \t\t\tif (repo_parse_commit(r, p)) {\n-\t\t\t\tclear_nonstale_queue(&queue);\n+\t\t\t\tclear_prio_queue(&state.queue);\n \t\t\t\tcommit_list_free(*result);\n \t\t\t\t*result = NULL;\n \t\t\t\t/*\n@@ -187,12 +254,11 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\treturn error(_(\"could not parse commit %s\"),\n \t\t\t\t\t oid_to_hex(&p->object.oid));\n \t\t\t}\n-\t\t\tp->object.flags |= flags;\n-\t\t\tnonstale_queue_put_dedup(&queue, p);\n+\t\t\tpaint_queue_put(&state, p, flags);\n \t\t}\n \t}\n \n-\tclear_nonstale_queue(&queue);\n+\tclear_prio_queue(&state.queue);\n \ttrace2_data_intmax(\"paint_down_to_common\", r,\n \t\t\t \"steps\", steps);\n \tcommit_list_sort_by_date(result);\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex b3a31b80ac..51f3d70492 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -289,7 +289,7 @@ test_expect_success 'get_merge_bases_many:pending-stale' '\n \t\tgit rev-parse ps-B\n \t} >expect &&\n \ttest_all_modes get_merge_bases_many &&\n-\ttest_paint_down_steps 6 6 6 6\n+\ttest_paint_down_steps 5 5 5 5\n '\n \n test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n@@ -304,7 +304,7 @@ test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n \t\tgit rev-parse pi-B\n \t} >expect &&\n \ttest_all_modes get_merge_bases_many &&\n-\ttest_paint_down_steps 5 5 5 5\n+\ttest_paint_down_steps 5 4 5 5\n '\n \n test_expect_success 'setup mixed finite/INFINITY topology' '\n-- \ngitgitgadget\n\n"},{"id":"546606","messageId":"4db485b48aae810eeba28ea4feb47401ab352e88.1782649547.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v4 6/8] commit-reach: remove unused nonstale_queue dedup wrappers","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:44Z","receivedAt":"2026-06-28T12:26:05Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nnonstale_queue_put_dedup() and nonstale_queue_get_dedup() became\nunused after the previous commit. The core nonstale_queue functions\nremain in use by ahead_behind().\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 18 ------------------\n 1 file changed, 18 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 9ae306f60c..176ffd68d0 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -79,24 +79,6 @@ static void clear_nonstale_queue(struct nonstale_queue *queue)\n \tqueue->max_nonstale = NULL;\n }\n \n-static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n-\t\t\t\t struct commit *c)\n-{\n-\tif (c->object.flags & ENQUEUED)\n-\t\treturn;\n-\tc->object.flags |= ENQUEUED;\n-\tnonstale_queue_put(queue, c);\n-}\n-\n-static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n-{\n-\tstruct commit *commit = nonstale_queue_get(queue);\n-\n-\tif (commit)\n-\t\tcommit->object.flags &= ~ENQUEUED;\n-\treturn commit;\n-}\n-\n /*\n * Priority queue with per-side commit counters for paint_down_to_common().\n * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n-- \ngitgitgadget\n\n"},{"id":"546607","messageId":"450678064972beae5f21b9135e34069a1a8ea585.1782649547.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v4 7/8] commit-reach: terminate merge-base walk when one paint side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:45Z","receivedAt":"2026-06-28T12:26:08Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd an early termination check to paint_down_to_common() using the\nper-side counters introduced earlier. Once the walk enters the\nfinite-generation region, terminate early when one side's exclusive\ncount drops to zero -- no new merge-base can form without both paint\nsides meeting.\n\nThe check also waits for pending_merge_bases to reach zero, ensuring\nall merge-base candidates have been dequeued and recorded before\nexiting.\n\nThe INFINITY gate ensures correctness: commits without a commit-graph\nentry have GENERATION_NUMBER_INFINITY and are ordered by commit date,\nwhich is not topologically reliable. The optimization only fires\nonce the walk enters the finite-generation region where ordering\nguarantees hold.\n\nStep counts measured with trace2 on git.git with commit-graph:\n\n merge-base --all v2.0.0 v2.55.0-rc1:\n before: 72264 steps after: 44589 steps\n\n merge-base --all v2.55.0-rc1 v2.55.0-rc1~5:\n before: 110 steps after: 7 steps\n\nHelped-by: Derrick Stolee \nHelped-by: Elijah Newren \nSigned-off-by: Kristofer Karlsson \n---\n .../technical/paint-down-to-common.adoc | 17 +++++++++++++++++\n commit-reach.c | 17 ++++++++++++++---\n t/t6600-test-reach.sh | 4 ++--\n 3 files changed, 33 insertions(+), 5 deletions(-)\n\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nindex ac3e2b39a5..15adac7885 100644\n--- a/Documentation/technical/paint-down-to-common.adoc\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -99,6 +99,9 @@ ends when one of the following conditions holds:\n 4. Single result: the caller only needs one merge base, one has\n been found, and the walk has entered the finite-generation\n region.\n+ 5. Side exhaustion: no pure PARENT1 or pure PARENT2 commits\n+ remain in the queue, no pending merge-base candidates exist,\n+ and the walk has entered the finite-generation region.\n \n Stale entry condition\n ~~~~~~~~~~~~~~~~~~~~~\n@@ -109,6 +112,20 @@ existing candidates by proving one is an ancestor of another, but\n `remove_redundant()` handles that as a post-processing step, so it\n is safe to exit early.\n \n+Side-exhaustion condition\n+~~~~~~~~~~~~~~~~~~~~~~~~~\n+A new merge-base requires commits from both sides to meet. When one\n+side's exclusive counter reaches zero and there are no pending\n+merge-base candidates, no future traversal step can produce a new\n+candidate.\n+\n+This optimization only activates in the finite-generation region\n+where topological ordering holds. In that region, children are\n+always visited before parents, so paint flags are final at visit\n+time and an exhausted side cannot reappear. In the INFINITY region,\n+commit-date ordering can violate this guarantee, so the check is\n+skipped.\n+\n Generation cutoff\n ~~~~~~~~~~~~~~~~~\n Some callers (notably `remove_redundant()`) supply a `min_generation`\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 176ffd68d0..e174b219c6 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -131,6 +131,10 @@ static void paint_queue_put(struct paint_state *state,\n \t}\n }\n \n+/*\n+ * Dequeue the next commit for the paint walk, or return NULL when\n+ * no more merge bases can be discovered.\n+ */\n static struct commit *paint_queue_get(struct paint_state *state)\n {\n \tstruct commit *commit = prio_queue_get(&state->queue);\n@@ -140,9 +144,16 @@ static struct commit *paint_queue_get(struct paint_state *state)\n \n \tcommit->object.flags &= ~ENQUEUED;\n \n-\tif (!state->parent1_count && !state->parent2_count &&\n-\t !state->mb_candidate_count)\n-\t\treturn NULL;\n+\tif (!state->mb_candidate_count) {\n+\t\t/* only stale entries remain */\n+\t\tif (!state->parent1_count && !state->parent2_count)\n+\t\t\treturn NULL;\n+\n+\t\t/* one side is exhausted */\n+\t\tif ((!state->parent1_count || !state->parent2_count) &&\n+\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n+\t\t\treturn NULL;\n+\t}\n \n \tpaint_count_update(state, commit->object.flags, -1);\n \treturn commit;\ndiff --git a/t/t6600-test-reach.sh b/t/t6600-test-reach.sh\nindex 51f3d70492..6365007560 100755\n--- a/t/t6600-test-reach.sh\n+++ b/t/t6600-test-reach.sh\n@@ -220,7 +220,7 @@ test_expect_success 'in_merge_bases_many:self' '\n \tEOF\n \techo \"in_merge_bases_many(A,X):1\" >expect &&\n \ttest_all_modes in_merge_bases_many &&\n-\ttest_paint_down_steps 45 2 25 3\n+\ttest_paint_down_steps 45 1 25 1\n '\n \n test_expect_success 'is_descendant_of:hit' '\n@@ -337,7 +337,7 @@ test_expect_success 'merge-base --all commit-walk steps' '\n \t>input &&\n \tgit rev-parse commit-9-1 >expect &&\n \trun_all_modes git merge-base --all commit-9-9 commit-9-1 &&\n-\ttest_paint_down_steps 81 80 81 81\n+\ttest_paint_down_steps 81 9 57 10\n '\n \n test_expect_success 'reduce_heads' '\n-- \ngitgitgadget\n\n"},{"id":"546608","messageId":"8dd15d44e6a60fc39bbf6d894628507e839f9248.1782649547.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v4 8/8] commit-reach: move min_generation check into paint_queue_get()","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-06-28T12:25:46Z","receivedAt":"2026-06-28T12:26:10Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nConsolidate the min_generation termination condition into\npaint_queue_get(), alongside the existing stale-entry and\nside-exhaustion checks.\n\nMove last_gen into struct paint_state so that\ncommit_graph_generation() is called exactly once per dequeued commit\nand the result is shared across all termination checks and the\nmonotonicity BUG assertion.\n\nSigned-off-by: Kristofer Karlsson \n---\n commit-reach.c | 31 ++++++++++++++++++-------------\n 1 file changed, 18 insertions(+), 13 deletions(-)\n\ndiff --git a/commit-reach.c b/commit-reach.c\nindex e174b219c6..5c5c54d66e 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -89,6 +89,8 @@ struct paint_state {\n \tsize_t parent1_count;\n \tsize_t parent2_count;\n \tsize_t mb_candidate_count;\n+\ttimestamp_t min_generation;\n+\ttimestamp_t last_gen;\n };\n \n static void paint_count_update(struct paint_state *state,\n@@ -138,11 +140,23 @@ static void paint_queue_put(struct paint_state *state,\n static struct commit *paint_queue_get(struct paint_state *state)\n {\n \tstruct commit *commit = prio_queue_get(&state->queue);\n+\ttimestamp_t generation;\n \n \tif (!commit)\n \t\treturn NULL;\n \n \tcommit->object.flags &= ~ENQUEUED;\n+\tgeneration = commit_graph_generation(commit);\n+\n+\tif (state->min_generation && generation > state->last_gen)\n+\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n+\t\t generation, state->last_gen,\n+\t\t oid_to_hex(&commit->object.oid));\n+\tstate->last_gen = generation;\n+\n+\t/* generation cutoff */\n+\tif (generation < state->min_generation)\n+\t\treturn NULL;\n \n \tif (!state->mb_candidate_count) {\n \t\t/* only stale entries remain */\n@@ -151,7 +165,7 @@ static struct commit *paint_queue_get(struct paint_state *state)\n \n \t\t/* one side is exhausted */\n \t\tif ((!state->parent1_count || !state->parent2_count) &&\n-\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n+\t\t generation < GENERATION_NUMBER_INFINITY)\n \t\t\treturn NULL;\n \t}\n \n@@ -177,9 +191,10 @@ static int paint_down_to_common(struct repository *r,\n \tstruct commit *commit;\n \tint i;\n \tint steps = 0;\n-\ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n+\tstate.min_generation = min_generation;\n+\tstate.last_gen = GENERATION_NUMBER_INFINITY;\n \tif (!min_generation && !corrected_commit_dates_enabled(r))\n \t\tstate.queue.compare = compare_commits_by_commit_date;\n \n@@ -196,18 +211,8 @@ static int paint_down_to_common(struct repository *r,\n \twhile ((commit = paint_queue_get(&state))) {\n \t\tstruct commit_list *parents;\n \t\tint flags;\n-\t\ttimestamp_t generation = commit_graph_generation(commit);\n \t\tsteps++;\n \n-\t\tif (min_generation && generation > last_gen)\n-\t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n-\t\t\t generation, last_gen,\n-\t\t\t oid_to_hex(&commit->object.oid));\n-\t\tlast_gen = generation;\n-\n-\t\tif (generation < min_generation)\n-\t\t\tbreak;\n-\n \t\tflags = commit->object.flags & (PARENT1 | PARENT2 | STALE);\n \t\tif (flags == (PARENT1 | PARENT2)) {\n \t\t\tif (!(commit->object.flags & RESULT)) {\n@@ -219,7 +224,7 @@ static int paint_down_to_common(struct repository *r,\n \t\t\t\t * descendant of this one.\n \t\t\t\t */\n \t\t\t\tif (!(mb_flags & MERGE_BASE_FIND_ALL) &&\n-\t\t\t\t generation < GENERATION_NUMBER_INFINITY)\n+\t\t\t\t state.last_gen < GENERATION_NUMBER_INFINITY)\n \t\t\t\t\tbreak;\n \t\t\t}\n \t\t\t/* Mark parents of a found merge stale */\n-- \ngitgitgadget\n"},{"id":"546610","messageId":"f77c0834-a001-400d-a8b8-a1e2398bf574@gmail.com","threadId":"65846","inReplyTo":"8dd15d44e6a60fc39bbf6d894628507e839f9248.1782649547.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 8/8] commit-reach: move min_generation check into paint_queue_get()","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-28T15:15:35Z","receivedAt":"2026-06-28T15:15:38Z","isPatch":true,"body":"On 6/28/26 8:25 AM, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n...> @@ -138,11 +140,23 @@ static void paint_queue_put(struct paint_state *state,\n> static struct commit *paint_queue_get(struct paint_state *state)\n> {\n> \tstruct commit *commit = prio_queue_get(&state->queue);\n> +\ttimestamp_t generation;\n> \n> \tif (!commit)\n> \t\treturn NULL;\n> \n> \tcommit->object.flags &= ~ENQUEUED;\n> +\tgeneration = commit_graph_generation(commit);\n> +\n> +\tif (state->min_generation && generation > state->last_gen)\n> +\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n> +\t\t generation, state->last_gen,\n> +\t\t oid_to_hex(&commit->object.oid));\n> +\tstate->last_gen = generation;\n> +\n> +\t/* generation cutoff */\n> +\tif (generation < state->min_generation)\n> +\t\treturn NULL;\n\n...\n\n> -\t\tif (min_generation && generation > last_gen)\n> -\t\t\tBUG(\"bad generation skip %\"PRItime\" > %\"PRItime\" at %s\",\n> -\t\t\t generation, last_gen,\n> -\t\t\t oid_to_hex(&commit->object.oid));\n> -\t\tlast_gen = generation;\n> -\n> -\t\tif (generation < min_generation)\n> -\t\t\tbreak;\n\nI'm just stopping in to say that this looks like a clean code move\nin this version, without mutating this chunk in the previous patch.\n\nLGTM.\n-Stolee\n"},{"id":"546611","messageId":"48bfdb11-2624-4aa6-8fbd-d3f894c33bcc@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-28T15:16:29Z","receivedAt":"2026-06-28T15:16:31Z","isPatch":true,"body":"On 6/28/26 8:25 AM, Kristofer Karlsson via GitGitGadget wrote:\n> commit-reach: terminate merge-base walk when one paint side is exhausted\n> \n> Optimize paint_down_to_common() for merge-base queries that hit large\n> one-sided histories.\n> \n> When the walk from one side reaches a commit with a very low generation\n> number that the other side never paints, the walk is forced to drain most of\n> the graph. A common trigger is a repository import that grafts a separate\n> history with its own root, but any merge that introduces a low-generation\n> commit never painted by the other side has the same effect.\n\n> Changes since v3:\n> \n> * Fixed BUG assertion that was accidentally made unconditional in v3:\n> restored the min_generation guard so it only fires when generation-based\n> ordering is active.\n> \n> * Moved generation cutoff and single-result termination conditions into the\n> documentation in patch 1/8, since they describe existing behavior.\n> \n> * Renamed paint_state counter fields for clarity: p1_count ->\n> parent1_count, p2_count -> parent2_count, pending_merge_bases ->\n> mb_candidate_count. Changed counter types from int to size_t. (Suggested\n> by Rene Scharfe.)\n\nI reviewed the v3 discussion, the range-diff, and reread patch 8. I think\nthat this version is good to go.\n\nThanks for your hard work!\n-Stolee\n\n"},{"id":"546619","messageId":"akIBvWT7nIWntCNT@szeder.dev","threadId":"65846","inReplyTo":"4db485b48aae810eeba28ea4feb47401ab352e88.1782649547.git.gitgitgadget@gmail.com","subject":"Re: [PATCH v4 6/8] commit-reach: remove unused nonstale_queue dedup wrappers","fromName":"SZEDER Gábor","fromEmail":"szeder.dev@gmail.com","sentAt":"2026-06-29T05:25:17Z","receivedAt":"2026-06-29T05:25:20Z","isPatch":true,"body":"On Sun, Jun 28, 2026 at 12:25:44PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> From: Kristofer Karlsson \n> \n> nonstale_queue_put_dedup() and nonstale_queue_get_dedup() became\n> unused after the previous commit. The core nonstale_queue functions\n> remain in use by ahead_behind().\n\nPlease squash this patch into the previous one. Since the last\ncallers of these static functions went away in that commit, it can't\nbe built with DEVELOPER=1:\n\n commit-reach.c:91:23: warning: ‘nonstale_queue_get_dedup’ defined but not used [-Wunused-function]\n 91 | static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n | ^~~~~~~~~~~~~~~~~~~~~~~~\n commit-reach.c:82:13: warning: ‘nonstale_queue_put_dedup’ defined but not used [-Wunused-function]\n 82 | static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n | ^~~~~~~~~~~~~~~~~~~~~~~~\n\n> Signed-off-by: Kristofer Karlsson \n> ---\n> commit-reach.c | 18 ------------------\n> 1 file changed, 18 deletions(-)\n> \n> diff --git a/commit-reach.c b/commit-reach.c\n> index 9ae306f60c..176ffd68d0 100644\n> --- a/commit-reach.c\n> +++ b/commit-reach.c\n> @@ -79,24 +79,6 @@ static void clear_nonstale_queue(struct nonstale_queue *queue)\n> \tqueue->max_nonstale = NULL;\n> }\n> \n> -static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n> -\t\t\t\t struct commit *c)\n> -{\n> -\tif (c->object.flags & ENQUEUED)\n> -\t\treturn;\n> -\tc->object.flags |= ENQUEUED;\n> -\tnonstale_queue_put(queue, c);\n> -}\n> -\n> -static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n> -{\n> -\tstruct commit *commit = nonstale_queue_get(queue);\n> -\n> -\tif (commit)\n> -\t\tcommit->object.flags &= ~ENQUEUED;\n> -\treturn commit;\n> -}\n> -\n> /*\n> * Priority queue with per-side commit counters for paint_down_to_common().\n> * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n> -- \n> gitgitgadget\n> \n"},{"id":"546660","messageId":"CAL71e4N92t8170UBW3rMA6B-rEUeOm-R_HSioB957mUKOpwRyQ@mail.gmail.com","threadId":"65846","inReplyTo":"akIBvWT7nIWntCNT@szeder.dev","subject":"Re: [PATCH v4 6/8] commit-reach: remove unused nonstale_queue dedup wrappers","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-29T10:09:26Z","receivedAt":"2026-06-29T10:09:39Z","isPatch":true,"body":"On Mon, 29 Jun 2026 at 07:25, SZEDER Gábor wrote:\n>\n> On Sun, Jun 28, 2026 at 12:25:44PM +0000, Kristofer Karlsson via GitGitGadget wrote:\n> > From: Kristofer Karlsson \n> >\n> > nonstale_queue_put_dedup() and nonstale_queue_get_dedup() became\n> > unused after the previous commit. The core nonstale_queue functions\n> > remain in use by ahead_behind().\n>\n> Please squash this patch into the previous one. Since the last\n> callers of these static functions went away in that commit, it can't\n> be built with DEVELOPER=1:\n>\n> commit-reach.c:91:23: warning: ‘nonstale_queue_get_dedup’ defined but not used [-Wunused-function]\n> 91 | static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n> | ^~~~~~~~~~~~~~~~~~~~~~~~\n> commit-reach.c:82:13: warning: ‘nonstale_queue_put_dedup’ defined but not used [-Wunused-function]\n> 82 | static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n> | ^~~~~~~~~~~~~~~~~~~~~~~~\n>\n\nThanks, will squash for v5! It's unfortunate that this means the commit itself\nbecomes less clean, but I don't have any other good solution\n-- and having each commit compile cleanly is more important.\n\n- Kristofer\n"},{"id":"546662","messageId":"CAL71e4O8gTLm4WUcPF-ZbOYTuEzuNSVh0Qjf8ys1w4LVF9Hi8Q@mail.gmail.com","threadId":"65846","inReplyTo":"48bfdb11-2624-4aa6-8fbd-d3f894c33bcc@gmail.com","subject":"Re: [PATCH v4 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-29T12:11:51Z","receivedAt":"2026-06-29T12:12:04Z","isPatch":true,"body":"On Sun, 28 Jun 2026 at 17:16, Derrick Stolee wrote:\n>\n> I reviewed the v3 discussion, the range-diff, and reread patch 8. I think\n> that this version is good to go.\n\nThanks for all your reviews and feedback. However, I found one more\nproblem that needs to be resolved before this is good to go.\n\npaint_down_to_common() has this fallback:\n\n if (!min_generation && !corrected_commit_dates_enabled(r))\n queue.pq.compare = compare_commits_by_commit_date;\n\nWhen this fires, the queue uses commit-date ordering instead of\ngeneration ordering. The side-exhaustion optimization and my older\npatch for !FIND_ALL early exit both check for reaching the finite\ngeneration, but with date ordering, that check is wrong --\na commit can have a finite topo level (it is in a v1 commit graph)\nwhile the queue is not ordered by generation. This unfortunately\nmeans there is a regression for the !FIND_ALL optimization that\nI should fix before 2.55 is final. I will send a small patch for\nthat separately: add tests that demonstrate the problem, and disable\nthe !FIND_ALL early exit when generation ordering is not active.\n\nI traced the history of this fallback. The queue was switched from\ndate ordering to generation ordering in 3afc679b (2018-05). Then in\n091f4cf3 (2018-08) you added the date fallback after finding that v1\ntopo levels caused \"git merge-base v4.8 v4.9\" on the Linux kernel to\nwalk 636k commits instead of 167k -- a side branch with a low topo\nlevel stayed in the queue behind a long chain, preventing early STALE\npropagation. Later, 8d00d7c3 (2021-01) tightened the fallback to\nonly fire without corrected commit dates, since v2 does not have the\nregression.\n\nThe problem that 091f4cf3 addresses looks closely related to what\nside-exhaustion solves: the walk goes deep into a subgraph where\nonly one paint side has presence. With side-exhaustion, the walk\nterminates as soon as one paint side is exhausted from the queue,\nso the deep walk never happens regardless of queue ordering.\n\nI benchmarked \"git merge-base --all v4.8 v4.9\" on the Linux kernel\n(the same case from 091f4cf3) with three configurations:\n\n master (--all) side-exhaust (--all, gen ordering)\n no graph: 3212 ms 3268 ms\n v1 graph: 188 ms 17 ms\n v2 graph: 227 ms 17 ms\n\nWith side-exhaustion, the v1 case no longer shows a regression\ncompared to the date fallback -- if anything, it is slightly faster\nsince the walk terminates earlier. This suggests that the workaround\nfrom 091f4cf3 may no longer be needed when side-exhaustion is\npresent.\n\nIt is also worth noting that commitGraph.generationVersion has\ndefaulted to 2 since 2021, so the v1 fallback path is rarely\nexercised in practice. Any commit-graph rewrite produces v2 data,\nand only repos that have not rewritten their commit graph in over\nfour years would still have v1-only data.\n\nIf that reasoning holds, the fix for v5 would be to remove the date\nfallback entirely, always using compare_commits_by_gen_then_commit_date.\nThis would:\n\n 1. Fix the bug (finite generation always means generation-ordered\n queue).\n 2. Remove corrected_commit_dates_enabled() which has no other\n callers.\n\nThe alternative would be to keep the fallback and disable the\noptimizations that depend on ordering (via a flag like\npaint_state.gen_ordered).\n\nDo you see any cases I might be missing where removing the fallback\ncould cause problems?\n\nThanks,\nKristofer\n"},{"id":"546663","messageId":"5ef694a3-9164-4ab4-8835-136439f6d267@gmail.com","threadId":"65846","inReplyTo":"CAL71e4O8gTLm4WUcPF-ZbOYTuEzuNSVh0Qjf8ys1w4LVF9Hi8Q@mail.gmail.com","subject":"Re: [PATCH v4 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Derrick Stolee","fromEmail":"stolee@gmail.com","sentAt":"2026-06-29T12:40:22Z","receivedAt":"2026-06-29T12:40:26Z","isPatch":true,"body":"On 6/29/2026 8:11 AM, Kristofer Karlsson wrote:\n> On Sun, 28 Jun 2026 at 17:16, Derrick Stolee wrote:\n>>\n>> I reviewed the v3 discussion, the range-diff, and reread patch 8. I think\n>> that this version is good to go.\n> \n> Thanks for all your reviews and feedback. However, I found one more\n> problem that needs to be resolved before this is good to go.\n> \n> paint_down_to_common() has this fallback:\n> \n> if (!min_generation && !corrected_commit_dates_enabled(r))\n> queue.pq.compare = compare_commits_by_commit_date;\n...> I traced the history of this fallback. \n...> The problem that 091f4cf3 addresses looks closely related to what\n> side-exhaustion solves: the walk goes deep into a subgraph where\n> only one paint side has presence. With side-exhaustion, the walk\n> terminates as soon as one paint side is exhausted from the queue,\n> so the deep walk never happens regardless of queue ordering.\n> \n> I benchmarked \"git merge-base --all v4.8 v4.9\" on the Linux kernel\n> (the same case from 091f4cf3) with three configurations:\n> \n> master (--all) side-exhaust (--all, gen ordering)\n> no graph: 3212 ms 3268 ms\n> v1 graph: 188 ms 17 ms\n> v2 graph: 227 ms 17 ms\n> \n> With side-exhaustion, the v1 case no longer shows a regression\n> compared to the date fallback -- if anything, it is slightly faster\n> since the walk terminates earlier. This suggests that the workaround\n> from 091f4cf3 may no longer be needed when side-exhaustion is\n> present.\n\nThanks for digging into the history of this fallback and catching it\nduring review!\n\n> If that reasoning holds, the fix for v5 would be to remove the date\n> fallback entirely, always using compare_commits_by_gen_then_commit_date.\n> This would:\n> \n> 1. Fix the bug (finite generation always means generation-ordered\n> queue).\n> 2. Remove corrected_commit_dates_enabled() which has no other\n> callers.\n\nI agree with your reasoning, data-backed discovery, and the course of\naction to fix this. I'm happy that you're able to close the loop on\nthis long-standing performance issue even with v1 generation numbers.\n> Do you see any cases I might be missing where removing the fallback\n> could cause problems?\nI don't see any other concerns here. You're right that if we were to\nhave a different mode that changes the priority-queue ordering, then\nthe side-exhaustion optimization cannot be trusted, but you will\nremove this possibility.\n\nIt _may_ be worth mentioning this with a comment when initializing\nthe queue order for the paint_queue, because the use of the queue\nrequires topological ordering.\n\nThanks,\n-Stolee\n\n"},{"id":"546664","messageId":"CAL71e4PpxGMsZLQPasECy5Z89EQPoOtC4LrLb8VyAo-2oabXyg@mail.gmail.com","threadId":"65846","inReplyTo":"5ef694a3-9164-4ab4-8835-136439f6d267@gmail.com","subject":"Re: [PATCH v4 0/8] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson","fromEmail":"krka@spotify.com","sentAt":"2026-06-29T12:59:18Z","receivedAt":"2026-06-29T12:59:30Z","isPatch":true,"body":"On Mon, 29 Jun 2026 at 14:40, Derrick Stolee wrote:\n>\n> I agree with your reasoning, data-backed discovery, and the course of\n> action to fix this. I'm happy that you're able to close the loop on\n> this long-standing performance issue even with v1 generation numbers.\n\nSounds good, then I can continue with the approach of removing some code\n(even though it will likely be a net addition in the end).\n\n> > Do you see any cases I might be missing where removing the fallback\n> > could cause problems?\n> I don't see any other concerns here. You're right that if we were to\n> have a different mode that changes the priority-queue ordering, then\n> the side-exhaustion optimization cannot be trusted, but you will\n> remove this possibility.\n>\n> It _may_ be worth mentioning this with a comment when initializing\n> the queue order for the paint_queue, because the use of the queue\n> requires topological ordering.\n\nYes my plan is to rewrite v5 in a few ways:\n- update original documentation to note that infinite -> finite\n generation does not always hold\n- add a test (or more than one) for this problem\n- don't introduce the bug at any point\n- add a commit to replace the disabled optimization with\n removal of the commit-date based ordering (+ doc update)\n\nThanks for helping with this,\nKristofer\n"},{"id":"546913","messageId":"pull.2149.v5.git.1782923832.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v4.git.1782649547.gitgitgadget@gmail.com","subject":"[PATCH v5 00/10] commit-reach: terminate merge-base walk when one side is exhausted","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-07-01T16:37:01Z","receivedAt":"2026-07-01T16:37:15Z","isPatch":true,"body":"Optimize paint_down_to_common() for merge-base queries that hit large\none-sided histories.\n\nWhen the walk from one side reaches a commit with a very low generation\nnumber that the other side never paints, the walk is forced to drain most of\nthe graph. A common trigger is a repository import that grafts a separate\nhistory with its own root, but any merge that introduces a low-generation\ncommit never painted by the other side has the same effect.\n\nA new merge-base candidate can only be discovered when exclusive PARENT1 and\nPARENT2 paint meet. This series teaches paint_down_to_common() to stop as\nsoon as one side has no exclusive commits left in the queue; once one side\nis exhausted, no further candidates can appear.\n\norigin/HEAD o o PR HEAD\n | |\n (import) o :\n / \\ /\n | o merge-base\n | |\n : : (~2.5M commits)\n | |\nimport root main root\n\n\nIn the RFC thread [1], Derrick Stolee provided a criss-cross counterexample\nthat sharpened the halt condition, and Elijah Newren independently\ndiscovered the same optimization and shared an implementation in PR #2150\n[2]. Patch 3 incorporates test cases from Elijah's branch.\n\nThis series implements the optimization only after the walk enters the\nfinite-generation region, where generation ordering guarantees that paint on\nvisited commits is final.\n\nPatch 2 adds a test_trace2_data_singular helper to test-lib-functions.sh\nthat reports expected/actual values on assertion failure instead of a silent\ngrep exit. This was invaluable during development for iterating on step\ncounts across the series, and should be valuable for repairing tests after\nfuture algorithmic changes. Happy to drop it if it is considered unnecessary\ninfrastructure.\n\nThe final patch removes the commit-date ordering fallback introduced by\n091f4cf3 (commit: don't use generation numbers if not needed, 2018-08-30).\nWith side-exhaustion in place, the fallback is no longer needed for\nperformance, and removing it ensures the queue is always generation-ordered\nregardless of graph version, so every termination condition can rely on a\nsingle ordering invariant. This patch can be dropped if the scope is too\nbroad for this series.\n\nNOTE: If the final patch is kept, the separate \"commit-reach: fix !FIND_ALL\nearly exit with v1 commit graph\" topic becomes unnecessary. Either way, the\ntwo topics conflict trivially and I am happy to reroll whichever lands\nsecond.\n\nBenchmarks\n\nTrace2 step counts are deterministic (measured via trace2_data_intmax added\nin patch 5). Wall-clock times are best-of-11 runs.\n\n2.6M-commit monorepo with commit-graph:\n\n steps wall-clock\nmerge-base --all (across import) 2143438 -> 3 3.67s -> 5ms\nmerge-base --all (1000 apart) 2692915 -> 1035 4.41s -> 7ms\nmerge-base --all (5000 apart) 2692915 -> 6401 4.45s -> 13ms\nmerge-base --all (HEAD vs import) 2698872 -> 45960 4.50s -> 79ms\nmerge-tree (across import) 2143438 -> 3 4.42s -> 11ms\n\n\ngit.git (88k commits, commit-graph):\n\n steps wall-clock\nmerge-base --all v2.0.0 v2.55.0-rc1 72264 -> 44589 110ms -> 68ms\nmerge-base --all HEAD HEAD~1000 9891 -> 3828 18ms -> 10ms\nmerge-base --all HEAD HEAD~10000 72303 -> 41487 101ms -> 50ms\n\n\nCC: Derrick Stolee stolee@gmail.com CC: Elijah Newren newren@gmail.com\n\n[1]\nhttps://lore.kernel.org/git/CAL71e4Ps-2_0+uuZu43N9pFnXBemoAohPs_eyRJf8taXHJPAXQ@mail.gmail.com/T/#u\n[2] https://github.com/gitgitgadget/git/pull/2150\n\nChanges since v4:\n\n * New patch 2/10: added test_trace2_data_singular helper to\n test-lib-functions.sh. Shows expected/actual values on assertion failure\n instead of a silent grep failure. Makes iterating on step counts much\n easier.\n\n * New patch 6/10: added clock-skew topologies (se-, se2-) that expose\n side-exhaustion bugs when the commit-date ordering fallback fires with a\n v1 commit graph. All topologies use a shared skew_commit helper. Includes\n step count assertions for edge-case tests from patch 3.\n\n * Folded the nonstale_queue dedup wrapper removal (previously separate\n patch 6/8) into the paint_state introduction in patch 7/10.\n\n * New patch 10/10: remove the commit-date ordering fallback in\n paint_down_to_common(). The fallback (091cf18e) was a performance\n optimization for v1 commit graphs, but it breaks the generation ordering\n invariant that both the side-exhaustion and single-result optimizations\n depend on. With side-exhaustion in place, the fallback is no longer\n needed. If kept, this supersedes the separate \"commit-reach: fix\n !FIND_ALL early exit with v1 commit graph\" topic.\n\nChanges since v3:\n\n * Fixed BUG assertion that was accidentally made unconditional in v3:\n restored the min_generation guard so it only fires when generation-based\n ordering is active.\n\n * Moved generation cutoff and single-result termination conditions into the\n documentation in patch 1, since they describe existing behavior.\n\n * Renamed paint_state counter fields for clarity: p1_count ->\n parent1_count, p2_count -> parent2_count, pending_merge_bases ->\n mb_candidate_count. Changed counter types from int to size_t. (Suggested\n by Rene Scharfe.)\n\nChanges since v2:\n\n * New patch 9/10 (was 8/8): moved the min_generation termination check and\n the last_gen monotonicity assertion into paint_queue_get(), consolidating\n halt conditions. commit_graph_generation() is now called once per\n dequeued commit and shared across all checks.\n\n * Moved all halt conditions inside paint_queue_get() with the \"pop first\"\n form: pop, check, then decrement counters. This keeps the optimization\n commit's diff minimal (just inserting the new checks between pop and\n decrement).\n\n * Shortened the doc comment on paint_queue_get() to describe what it does\n rather than how. Inline comments on each return NULL explain the specific\n halt condition.\n\n * Replaced the manual commit-graph setup in the step-count test with\n run_all_modes, which now sets GIT_TRACE2_EVENT per mode and produces\n trace-mode-{none,full,half,no-gdat}.txt files.\n\n * Added a test_paint_down_steps helper for concise 4-mode step assertions\n with diagnostic output on mismatch (prints \"expected X, got Y\" instead of\n a silent grep failure).\n\n * Added step-count assertions to the single-walk edge-case tests:\n in_merge_bases_many:self, pending-stale, infinity-both-sides,\n mixed-finite-infinity.\n\n * Included step counts alongside wall-clock times in the benchmark tables.\n\nChanges since v1:\n\n * Reordered patches: documentation first (describing the existing\n algorithm), tests before code changes, so they demonstrate passing with\n old logic first.\n\n * Dropped the ahead_behind decoupling patch. paint_state is now a NEW\n struct alongside nonstale_queue instead of replacing it. ahead_behind()\n is completely untouched.\n\n * Removed nonstale_queue_put_dedup() and nonstale_queue_get_dedup() (dead\n code after the conversion) in a separate commit.\n\n * Renamed: struct paint_queue -> paint_state, field pq -> queue,\n paint_count_add/remove -> paint_count_update (single function with signed\n delta parameter).\n\n * Split the old paint_count_transition (which handled both old and new\n flags in one call) into separate remove/add calls with a signed delta.\n This eliminates the need for the case 0 handler (which tracked \"not in\n the queue\") and allows an exhaustive switch on (PARENT1 | PARENT2 |\n STALE) that documents all valid flag combinations, with BUG() in default.\n\n * Added trace2_data_intmax() instrumentation to report the number of\n commits visited per paint walk (separate commit), with step-count\n assertions in tests for deterministic regression detection.\n\nElijah Newren (1):\n t6600: add test cases for side-exhaustion edge cases\n\nKristofer Karlsson (9):\n Documentation/technical: add paint-down-to-common doc\n test-lib-functions: improve diagnostic output for trace2 data\n assertions\n t6099, t6600: add side-exhaustion regression tests\n commit-reach: add trace2 instrumentation to paint_down_to_common()\n t6600: add clock-skew topologies and step counts for edge cases\n commit-reach: introduce struct paint_state with per-side counters\n commit-reach: terminate merge-base walk when one paint side is\n exhausted\n commit-reach: move min_generation check into paint_queue_get()\n commit-reach: remove commit-date ordering fallback\n\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 151 +++++++++\n commit-graph.c | 11 -\n commit-graph.h | 6 -\n commit-reach.c | 152 ++++++---\n t/meson.build | 1 +\n t/t6099-merge-base-side-exhaustion.sh | 82 +++++\n t/t6600-test-reach.sh | 289 +++++++++++++++++-\n t/test-lib-functions.sh | 36 +++\n 10 files changed, 663 insertions(+), 67 deletions(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n create mode 100755 t/t6099-merge-base-side-exhaustion.sh\n\n\nbase-commit: e9019fcafe0040228b8631c30f97ae1adb61bcdc\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-2149%2Fspkrka%2Fside-exhaust-pr-v5\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2149/spkrka/side-exhaust-pr-v5\nPull-Request: https://github.com/gitgitgadget/git/pull/2149\n\nRange-diff vs v4:\n\n 1: 3efb095b03 ! 1: be00f5aaa1 Documentation/technical: add paint-down-to-common doc\n @@ Documentation/technical/paint-down-to-common.adoc (new)\n + - PARENT1: reachable from `one`\n + - PARENT2: reachable from any commit in `twos[]`\n +\n -+The walk uses a priority queue ordered by generation number (falling\n -+back to commit date when generation numbers are unavailable). Each\n -+step dequeues the highest-priority commit (this is when we say a\n -+commit is \"visited\") and propagates its paint flags to its parents,\n -+enqueuing them if they gained new flags. When a commit receives\n -+both PARENT1 and PARENT2, it is a merge-base candidate. A candidate\n -+gains the STALE flag so its ancestors propagate staleness -- any\n -+deeper common ancestor is necessarily redundant.\n -+\n ++The walk uses a priority queue ordered by generation number\n ++(highest first), breaking ties by commit date. Each step dequeues\n ++the highest-priority commit (this is when we say a commit is\n ++\"visited\") and propagates its paint flags to its parents, enqueuing\n ++them if they gained new flags. When a commit receives both PARENT1\n ++and PARENT2, it is a merge-base candidate. A candidate gains the\n ++STALE flag so its ancestors propagate staleness -- any deeper common\n ++ancestor is necessarily redundant.\n ++\n ++NOTE: When the commit-graph uses only topological levels (generation\n ++number v1) and the caller passes `min_generation = 0`, a legacy\n ++fallback replaces the generation-ordered comparator with a pure\n ++commit-date comparator. This breaks the ordering invariants\n ++described below -- see <>.\n ++\n ++[[generation-regions]]\n +INFINITY and finite generation regions\n +--------------------------------------\n +\n ++The properties in this section assume generation-number ordering (the\n ++default comparator). They do NOT hold when the date-ordering fallback\n ++is active -- see <>.\n ++\n +The commit-graph stores a generation number for each commit. Commits\n +not in the commit-graph have generation `GENERATION_NUMBER_INFINITY`. The\n +graph is closed under reachability: if a commit is in the graph, all\n @@ Documentation/technical/paint-down-to-common.adoc (new)\n +\n +Single result\n +~~~~~~~~~~~~~\n -+When only one merge base is needed and the walk is in the\n -+finite-generation region, the first candidate found is necessarily\n -+the highest-generation common ancestor. No remaining commit in the\n -+queue can be a descendant of this candidate (generation ordering\n -+guarantees children are visited first), so it cannot be redundant\n -+and the walk can stop immediately.\n ++When only one merge base is needed, the walk is in the\n ++finite-generation region, and the queue uses generation ordering,\n ++the first candidate found is necessarily the highest-generation\n ++common ancestor. No remaining commit in the queue can be a\n ++descendant of this candidate (generation ordering guarantees\n ++children are visited first), so it cannot be redundant and the walk\n ++can stop immediately.\n ++\n ++This optimization is NOT safe when the date-ordering fallback is\n ++active, because commit-date order can visit a deeper ancestor\n ++before a shallower one -- see <>.\n ++\n ++[[date-ordering-fallback]]\n ++Date-ordering fallback\n ++----------------------\n ++\n ++When `min_generation` is zero and the commit-graph does not contain\n ++corrected commit dates (generation number v1, which stores only\n ++topological levels), `paint_down_to_common()` replaces the default\n ++generation-ordered comparator with `compare_commits_by_commit_date`.\n ++\n ++This was introduced as a performance heuristic: topological levels\n ++are coarser than commit dates, so date ordering can reach merge\n ++bases in fewer steps when timestamps are well-behaved. However,\n ++commit dates are not required to be monotonic -- a parent can have\n ++a later date than its child (clock skew, rebases, etc.) -- so the\n ++queue may visit commits out of topological order.\n ++\n ++This disables optimizations that depend on generation ordering:\n ++\n ++ 1. *Single result*: the first merge-base candidate found may not\n ++ be the shallowest, because a deeper ancestor with a higher\n ++ commit date can be dequeued first.\n ++\n ++ 2. *Side-exhaustion* (see subsequent commits): one paint side can\n ++ appear to drain from the queue while commits from that side are\n ++ still waiting with lower dates, causing premature termination.\n +\n +Related documentation\n +---------------------\n -: ---------- > 2: 143f5e0cc1 test-lib-functions: improve diagnostic output for trace2 data assertions\n 2: 1a0154b406 ! 3: eec4a9d926 t6600: add test cases for side-exhaustion edge cases\n @@ t/t6600-test-reach.sh: test_expect_success 'setup' '\n +\tpi_x=\"$(cat pi-X-oid)\" &&\n +\tgit branch -f pi-X-br \"$pi_x\" &&\n +\tgit tag pi-X \"$pi_x\" &&\n -+\n \tgit commit-graph write --reachable &&\n \tmv .git/objects/info/commit-graph commit-graph-full &&\n \tchmod u+w commit-graph-full &&\n 3: 017bf156c5 ! 4: 7f0efe6c73 t6099, t6600: add side-exhaustion regression tests\n @@ t/t6099-merge-base-side-exhaustion.sh (new)\n +'\n +\n +test_done\n -\n - ## t/t6600-test-reach.sh ##\n -@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n - \ttest_all_modes get_merge_bases_many\n - '\n - \n -+test_expect_success 'setup mixed finite/INFINITY topology' '\n -+\t# Create a commit outside all saved commit-graph files so it always\n -+\t# has INFINITY generation, while its parent (ps-X) is in the graph\n -+\t# with a finite generation. Use the ps-* orphan topology so we do\n -+\t# not pollute the grid-based rev-list tests.\n -+\tgit checkout ps-X &&\n -+\ttest_env GIT_TEST_COMMIT_GRAPH= test_commit pm-INF\n -+'\n -+\n -+test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n -+\t# One tip (pm-INF) is outside the commit-graph with INFINITY\n -+\t# generation; the other (ps-B) is in the graph with finite\n -+\t# generation. The walk starts in the INFINITY region and crosses\n -+\t# into the finite region where side-exhaustion can fire.\n -+\tcat >input <<-\\EOF &&\n -+\tA:pm-INF\n -+\tX:ps-B\n -+\tEOF\n -+\t{\n -+\t\techo \"get_merge_bases_many(A,X):\" &&\n -+\t\tgit rev-parse ps-X\n -+\t} >expect &&\n -+\ttest_all_modes get_merge_bases_many\n -+'\n -+\n - test_expect_success 'reduce_heads' '\n - \tcat >input <<-\\EOF &&\n - \tX:commit-1-10\n 4: df3b090a2b ! 5: b9fb8bfe26 commit-reach: add trace2 instrumentation to paint_down_to_common()\n @@ t/t6600-test-reach.sh: test_expect_success 'setup' '\n +test_paint_down_steps () {\n +\tfor mode in none full half no-gdat\n +\tdo\n -+\t\ttest_trace2_data paint_down_to_common steps \"$1\" \\\n -+\t\t\t<\"trace-mode-${mode}.txt\" || return 1\n ++\t\ttest_trace2_data_singular paint_down_to_common steps \"$1\" \\\n ++\t\t\t\"mode=$mode\" <\"trace-mode-${mode}.txt\" || return 1\n +\t\tshift\n +\tdone\n +}\n @@ t/t6600-test-reach.sh: test_expect_success 'in_merge_bases_many:self' '\n '\n \n test_expect_success 'is_descendant_of:hit' '\n -@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:pending-stale' '\n - \t\techo \"get_merge_bases_many(A,X):\" &&\n - \t\tgit rev-parse ps-B\n - \t} >expect &&\n --\ttest_all_modes get_merge_bases_many\n -+\ttest_all_modes get_merge_bases_many &&\n -+\ttest_paint_down_steps 6 6 6 6\n - '\n - \n - test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n @@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:infinity-both-sides' '\n - \t\techo \"get_merge_bases_many(A,X):\" &&\n - \t\tgit rev-parse pi-B\n - \t} >expect &&\n --\ttest_all_modes get_merge_bases_many\n -+\ttest_all_modes get_merge_bases_many &&\n -+\ttest_paint_down_steps 5 5 5 5\n + \ttest_all_modes get_merge_bases_many\n '\n \n - test_expect_success 'setup mixed finite/INFINITY topology' '\n -@@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:mixed-finite-infinity' '\n - \t\techo \"get_merge_bases_many(A,X):\" &&\n - \t\tgit rev-parse ps-X\n - \t} >expect &&\n --\ttest_all_modes get_merge_bases_many\n -+\ttest_all_modes get_merge_bases_many &&\n -+\ttest_paint_down_steps 3 3 3 3\n -+'\n -+\n +test_expect_success 'merge-base --all commit-walk steps' '\n +\t>input &&\n +\tgit rev-parse commit-9-1 >expect &&\n +\trun_all_modes git merge-base --all commit-9-9 commit-9-1 &&\n +\ttest_paint_down_steps 81 80 81 81\n - '\n - \n ++'\n ++\n test_expect_success 'reduce_heads' '\n + \tcat >input <<-\\EOF &&\n + \tX:commit-1-10\n -: ---------- > 6: c6e3cc13f7 t6600: add clock-skew topologies and step counts for edge cases\n 5: fed9f2c368 ! 7: 42a75899f2 commit-reach: introduce struct paint_state with per-side counters\n @@ Commit message\n earlier than the old code in some topologies (the popped stale\n commit is never processed), so a few step counts drop by one.\n \n - The existing nonstale_queue is left in place for ahead_behind().\n + The existing nonstale_queue is left in place for ahead_behind(),\n + though nonstale_queue_put_dedup() and nonstale_queue_get_dedup()\n + became unused and are removed.\n \n Signed-off-by: Kristofer Karlsson \n \n @@ Documentation/technical/paint-down-to-common.adoc: re-enqueued is bounded by the\n 4. Single result: the caller only needs one merge base, one has\n \n ## commit-reach.c ##\n -@@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n - \treturn commit;\n +@@ commit-reach.c: static void clear_nonstale_queue(struct nonstale_queue *queue)\n + \tqueue->max_nonstale = NULL;\n }\n \n +-static void nonstale_queue_put_dedup(struct nonstale_queue *queue,\n +-\t\t\t\t struct commit *c)\n +/*\n + * Priority queue with per-side commit counters for paint_down_to_common().\n + * Each non-stale queued commit occupies exactly one bucket: PARENT1-only,\n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_q\n +\tsize_t parent1_count;\n +\tsize_t parent2_count;\n +\tsize_t mb_candidate_count;\n ++\tint gen_ordered;\n +};\n +\n +static void paint_count_update(struct paint_state *state,\n +\t\t\t unsigned flags, int delta)\n -+{\n + {\n +-\tif (c->object.flags & ENQUEUED)\n +-\t\treturn;\n +-\tc->object.flags |= ENQUEUED;\n +-\tnonstale_queue_put(queue, c);\n +\tswitch (flags & (PARENT1 | PARENT2 | STALE)) {\n +\tcase PARENT1:\n +\t\tstate->parent1_count += delta;\n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_q\n +\tdefault:\n +\t\tBUG(\"unexpected paint state\");\n +\t}\n -+}\n -+\n + }\n + \n +-static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n +static void paint_queue_put(struct paint_state *state,\n +\t\t\t struct commit *c, unsigned add_flags)\n -+{\n + {\n +-\tstruct commit *commit = nonstale_queue_get(queue);\n +\tunsigned old_flags = c->object.flags;\n +\tc->object.flags |= add_flags;\n +\n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_q\n +\t\tpaint_count_update(state, c->object.flags, 1);\n +\t}\n +}\n -+\n + \n +-\tif (commit)\n +-\t\tcommit->object.flags &= ~ENQUEUED;\n +static struct commit *paint_queue_get(struct paint_state *state)\n +{\n +\tstruct commit *commit = prio_queue_get(&state->queue);\n @@ commit-reach.c: static struct commit *nonstale_queue_get_dedup(struct nonstale_q\n +\t\treturn NULL;\n +\n +\tpaint_count_update(state, commit->object.flags, -1);\n -+\treturn commit;\n -+}\n -+\n - /*\n - * See Documentation/technical/paint-down-to-common.adoc\n - *\n + \treturn commit;\n + }\n + \n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n \t\t\t\tenum merge_base_flags mb_flags,\n \t\t\t\tstruct commit_list **result)\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n -\tstruct nonstale_queue queue = {\n -\t\t{ compare_commits_by_gen_then_commit_date }\n +\tstruct paint_state state = {\n -+\t\t.queue = { compare_commits_by_gen_then_commit_date }\n ++\t\t.queue = { compare_commits_by_gen_then_commit_date },\n ++\t\t.gen_ordered = 1,\n \t};\n +\tstruct commit *commit;\n \tint i;\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n \ttimestamp_t last_gen = GENERATION_NUMBER_INFINITY;\n \tstruct commit_list **tail = result;\n \n - \tif (!min_generation && !corrected_commit_dates_enabled(r))\n +-\tif (!min_generation && !corrected_commit_dates_enabled(r))\n -\t\tqueue.pq.compare = compare_commits_by_commit_date;\n ++\tif (!min_generation && !corrected_commit_dates_enabled(r)) {\n +\t\tstate.queue.compare = compare_commits_by_commit_date;\n ++\t\tstate.gen_ordered = 0;\n ++\t}\n \n \tone->object.flags |= PARENT1;\n \tif (!n) {\n @@ t/t6600-test-reach.sh: test_expect_success 'get_merge_bases_many:infinity-both-s\n '\n \n test_expect_success 'setup mixed finite/INFINITY topology' '\n +@@ t/t6600-test-reach.sh: test_expect_success 'merge-base --all with clock skew returns wrong merge base (\n + \t>input &&\n + \tgit rev-parse se2-MB1 >expect &&\n + \trun_all_modes git merge-base --all se2-A se2-B &&\n +-\ttest_paint_down_steps 8 7 8 8\n ++\ttest_paint_down_steps 8 6 8 8\n + '\n + \n + test_expect_success 'reduce_heads' '\n 6: 4db485b48a < -: ---------- commit-reach: remove unused nonstale_queue dedup wrappers\n 7: 4506780649 ! 8: f8f00b7bcc commit-reach: terminate merge-base walk when one paint side is exhausted\n @@ commit-reach.c: static struct commit *paint_queue_get(struct paint_state *state)\n +\n +\t\t/* one side is exhausted */\n +\t\tif ((!state->parent1_count || !state->parent2_count) &&\n ++\t\t state->gen_ordered &&\n +\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n +\t\t\treturn NULL;\n +\t}\n @@ t/t6600-test-reach.sh: test_expect_success 'merge-base --all commit-walk steps'\n \tgit rev-parse commit-9-1 >expect &&\n \trun_all_modes git merge-base --all commit-9-9 commit-9-1 &&\n -\ttest_paint_down_steps 81 80 81 81\n -+\ttest_paint_down_steps 81 9 57 10\n ++\ttest_paint_down_steps 81 9 57 81\n '\n \n - test_expect_success 'reduce_heads' '\n + test_expect_success 'merge-base --all with clock skew and v1 commit-graph (side-exhaustion)' '\n 8: 8dd15d44e6 ! 9: b45c302caa commit-reach: move min_generation check into paint_queue_get()\n @@ Commit message\n \n ## commit-reach.c ##\n @@ commit-reach.c: struct paint_state {\n - \tsize_t parent1_count;\n \tsize_t parent2_count;\n \tsize_t mb_candidate_count;\n + \tint gen_ordered;\n +\ttimestamp_t min_generation;\n +\ttimestamp_t last_gen;\n };\n @@ commit-reach.c: static void paint_queue_put(struct paint_state *state,\n \tif (!state->mb_candidate_count) {\n \t\t/* only stale entries remain */\n @@ commit-reach.c: static struct commit *paint_queue_get(struct paint_state *state)\n - \n \t\t/* one side is exhausted */\n \t\tif ((!state->parent1_count || !state->parent2_count) &&\n + \t\t state->gen_ordered &&\n -\t\t commit_graph_generation(commit) < GENERATION_NUMBER_INFINITY)\n +\t\t generation < GENERATION_NUMBER_INFINITY)\n \t\t\treturn NULL;\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n \n +\tstate.min_generation = min_generation;\n +\tstate.last_gen = GENERATION_NUMBER_INFINITY;\n - \tif (!min_generation && !corrected_commit_dates_enabled(r))\n + \tif (!min_generation && !corrected_commit_dates_enabled(r)) {\n \t\tstate.queue.compare = compare_commits_by_commit_date;\n - \n + \t\tstate.gen_ordered = 0;\n @@ commit-reach.c: static int paint_down_to_common(struct repository *r,\n \twhile ((commit = paint_queue_get(&state))) {\n \t\tstruct commit_list *parents;\n -: ---------- > 10: d68972b1d7 commit-reach: remove commit-date ordering fallback\n\n-- \ngitgitgadget\n"},{"id":"546914","messageId":"be00f5aaa163d18a36bfa399346370e03322bbe2.1782923832.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v5.git.1782923832.gitgitgadget@gmail.com","subject":"[PATCH v5 01/10] Documentation/technical: add paint-down-to-common doc","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-07-01T16:37:02Z","receivedAt":"2026-07-01T16:37:16Z","isPatch":true,"body":"From: Kristofer Karlsson \n\nAdd a technical document describing the paint_down_to_common()\nalgorithm used for merge-base computation, covering the paint\nwalk, generation number regions, and termination conditions.\n\nSigned-off-by: Kristofer Karlsson \n---\n Documentation/Makefile | 1 +\n Documentation/technical/meson.build | 1 +\n .../technical/paint-down-to-common.adoc | 177 ++++++++++++++++++\n commit-reach.c | 6 +-\n 4 files changed, 184 insertions(+), 1 deletion(-)\n create mode 100644 Documentation/technical/paint-down-to-common.adoc\n\ndiff --git a/Documentation/Makefile b/Documentation/Makefile\nindex 2699f0b24a..f8dea4b395 100644\n--- a/Documentation/Makefile\n+++ b/Documentation/Makefile\n@@ -129,6 +129,7 @@ TECH_DOCS += technical/long-running-process-protocol\n TECH_DOCS += technical/multi-pack-index\n TECH_DOCS += technical/packfile-uri\n TECH_DOCS += technical/pack-heuristics\n+TECH_DOCS += technical/paint-down-to-common\n TECH_DOCS += technical/parallel-checkout\n TECH_DOCS += technical/partial-clone\n TECH_DOCS += technical/platform-support\ndiff --git a/Documentation/technical/meson.build b/Documentation/technical/meson.build\nindex ec07088c57..9ce11d5e48 100644\n--- a/Documentation/technical/meson.build\n+++ b/Documentation/technical/meson.build\n@@ -18,6 +18,7 @@ articles = [\n 'multi-pack-index.adoc',\n 'packfile-uri.adoc',\n 'pack-heuristics.adoc',\n+ 'paint-down-to-common.adoc',\n 'parallel-checkout.adoc',\n 'partial-clone.adoc',\n 'platform-support.adoc',\ndiff --git a/Documentation/technical/paint-down-to-common.adoc b/Documentation/technical/paint-down-to-common.adoc\nnew file mode 100644\nindex 0000000000..ff015c5c8f\n--- /dev/null\n+++ b/Documentation/technical/paint-down-to-common.adoc\n@@ -0,0 +1,177 @@\n+Merge-Base Computation and paint_down_to_common()\n+==================================================\n+\n+The function `paint_down_to_common()` in `commit-reach.c` computes merge\n+bases by walking the commit graph backwards from two sets of tips and\n+finding where their ancestry meets.\n+\n+Use cases\n+---------\n+\n+Computing merge bases is used in two different ways:\n+\n+ 1. *Finding all merge bases* (`merge-base --all`, `merge-tree`,\n+ `merge`, `rebase`). A merge base is a common ancestor that is\n+ not itself an ancestor of another common ancestor.\n+\n+ 2. *Ancestry checks* (`in_merge_bases`, used by `merge-base\n+ --is-ancestor`, `branch -d`, `fetch`). These ask: \"is commit A\n+ an ancestor of commit B?\" If a common ancestor equals one of the\n+ inputs, that input is necessarily the only merge base -- no other\n+ common ancestor can be both as recent and not an ancestor of it.\n+\n+Both use cases share the same algorithm and implementation.\n+\n+Algorithm\n+---------\n+\n+Given a commit `one` and a set of commits `twos[]`, the walk paints\n+commits with two colors:\n+\n+ - PARENT1: reachable from `one`\n+ - PARENT2: reachable from any commit in `twos[]`\n+\n+The walk uses a priority queue ordered by generation number\n+(highest first), breaking ties by commit date. Each step dequeues\n+the highest-priority commit (this is when we say a commit is\n+\"visited\") and propagates its paint flags to its parents, enqueuing\n+them if they gained new flags. When a commit receives both PARENT1\n+and PARENT2, it is a merge-base candidate. A candidate gains the\n+STALE flag so its ancestors propagate staleness -- any deeper common\n+ancestor is necessarily redundant.\n+\n+NOTE: When the commit-graph uses only topological levels (generation\n+number v1) and the caller passes `min_generation = 0`, a legacy\n+fallback replaces the generation-ordered comparator with a pure\n+commit-date comparator. This breaks the ordering invariants\n+described below -- see <>.\n+\n+[[generation-regions]]\n+INFINITY and finite generation regions\n+--------------------------------------\n+\n+The properties in this section assume generation-number ordering (the\n+default comparator). They do NOT hold when the date-ordering fallback\n+is active -- see <>.\n+\n+The commit-graph stores a generation number for each commit. Commits\n+not in the commit-graph have generation `GENERATION_NUMBER_INFINITY`. The\n+graph is closed under reachability: if a commit is in the graph, all\n+its ancestors are too. This partitions the commit graph into two regions:\n+\n+....\n+ +---------------------------------------+\n+ | INFINITY region |\n+ | generation = INFINITY |\n+ | queue order: heuristic (commit date) |\n+ +---------------------------------------+\n+ |\n+ v\n+ +---------------------------------------+\n+ | Finite region |\n+ | generation = finite |\n+ | queue order: topological |\n+ +---------------------------------------+\n+....\n+\n+When the commit-graph is enabled, the INFINITY region is typically\n+very small -- it only contains commits added since the last\n+commit-graph refresh.\n+\n+All reachable INFINITY-generation commits are visited before any\n+finite-generation commit, because INFINITY is larger than any finite\n+value. Once the walk crosses into the finite region, it stays there.\n+\n+In the finite region, generation ordering guarantees topological\n+traversal: children are always visited before their parents. This\n+means that paint on already-visited commits is final -- no future\n+traversal step can add paint to them.\n+\n+In the INFINITY region, commit-date ordering can violate this: a\n+parent with a later date can be visited before a child with an earlier\n+date. Paint flags are therefore NOT final at visit time, and a\n+commit visited with only one side's paint may later gain the other.\n+\n+Paint flags are only added, never removed. Since each flag can be set\n+at most once per commit, the number of times a commit can be\n+re-enqueued is bounded by the number of flag transitions.\n+\n+Termination\n+-----------\n+\n+The walk uses a `nonstale_queue` wrapper around `prio_queue` that\n+tracks `max_nonstale`: the lowest-priority non-stale commit enqueued\n+so far. Once that commit is dequeued, every remaining entry is known\n+to be STALE and the loop terminates. Specifically, the main loop\n+ends when one of the following conditions holds:\n+\n+ 1. The queue is empty.\n+ 2. `max_nonstale` has been dequeued, meaning the queue only contains\n+ STALE entries.\n+ 3. Generation cutoff: the dequeued commit's generation is below\n+ a caller-supplied `min_generation` threshold.\n+ 4. Single result: the caller only needs one merge base, one has\n+ been found, and the walk has entered the finite-generation\n+ region.\n+\n+Stale entry condition\n+~~~~~~~~~~~~~~~~~~~~~\n+Once all queued entries are stale, no new merge-base candidates can\n+be discovered -- that requires at least one non-stale commit from\n+each side meeting. Continuing the walk could still invalidate\n+existing candidates by proving one is an ancestor of another, but\n+`remove_redundant()` handles that as a post-processing step, so it\n+is safe to exit early.\n+\n+Generation cutoff\n+~~~~~~~~~~~~~~~~~\n+Some callers (notably `remove_redundant()`) supply a `min_generation`\n+threshold -- the minimum generation of the input commits. No merge\n+base can have a generation below this threshold, so the walk\n+terminates as soon as it dequeues such a commit.\n+\n+Single result\n+~~~~~~~~~~~~~\n+When only one merge base is needed, the walk is in the\n+finite-generation region, and the queue uses generation ordering,\n+the first candidate found is necessarily the highest-generation\n+common ancestor. No remaining commit in the queue can be a\n+descendant of this candidate (generation ordering guarantees\n+children are visited first), so it cannot be redundant and the walk\n+can stop immediately.\n+\n+This optimization is NOT safe when the date-ordering fallback is\n+active, because commit-date order can visit a deeper ancestor\n+before a shallower one -- see <>.\n+\n+[[date-ordering-fallback]]\n+Date-ordering fallback\n+----------------------\n+\n+When `min_generation` is zero and the commit-graph does not contain\n+corrected commit dates (generation number v1, which stores only\n+topological levels), `paint_down_to_common()` replaces the default\n+generation-ordered comparator with `compare_commits_by_commit_date`.\n+\n+This was introduced as a performance heuristic: topological levels\n+are coarser than commit dates, so date ordering can reach merge\n+bases in fewer steps when timestamps are well-behaved. However,\n+commit dates are not required to be monotonic -- a parent can have\n+a later date than its child (clock skew, rebases, etc.) -- so the\n+queue may visit commits out of topological order.\n+\n+This disables optimizations that depend on generation ordering:\n+\n+ 1. *Single result*: the first merge-base candidate found may not\n+ be the shallowest, because a deeper ancestor with a higher\n+ commit date can be dequeued first.\n+\n+ 2. *Side-exhaustion* (see subsequent commits): one paint side can\n+ appear to drain from the queue while commits from that side are\n+ still waiting with lower dates, causing premature termination.\n+\n+Related documentation\n+---------------------\n+\n+ - `Documentation/technical/commit-graph.adoc` -- generation numbers\n+ and the reachability closure property.\ndiff --git a/commit-reach.c b/commit-reach.c\nindex 5df471a313..a9483759e0 100644\n--- a/commit-reach.c\n+++ b/commit-reach.c\n@@ -96,7 +96,11 @@ static struct commit *nonstale_queue_get_dedup(struct nonstale_queue *queue)\n \treturn commit;\n }\n \n-/* all input commits in one and twos[] must have been parsed! */\n+/*\n+ * See Documentation/technical/paint-down-to-common.adoc\n+ *\n+ * All input commits in one and twos[] must have been parsed!\n+ */\n static int paint_down_to_common(struct repository *r,\n \t\t\t\tstruct commit *one, int n,\n \t\t\t\tstruct commit **twos,\n-- \ngitgitgadget\n\n"},{"id":"546915","messageId":"143f5e0cc183bff1bbc5e9cfa65f456f615f4ad0.1782923832.git.gitgitgadget@gmail.com","threadId":"65846","inReplyTo":"pull.2149.v5.git.1782923832.gitgitgadget@gmail.com","subject":"[PATCH v5 02/10] test-lib-functions: improve diagnostic output for trace2 data assertions","fromName":"Kristofer Karlsson via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2026-07-01T16:37:03Z","receivedAt":"2026-07-01T16:37:18Z","isPatch":true,"body":"From: Kristofer Karlsson \n\ntest_trace2_data is a bare grep that silently exits on failure.\nAdd a more informative variant that verifies the event appears\nexactly once and reports what went wrong: key not found, multiple\nentries, or value mismatch. Diagnostics go to FD 4 like test_grep.\n\nBefore (value mismatch):\n\n $ test_trace2_data status count/changed 999 \n---\n t/test-lib-functions.sh | 36 ++++++++++++++++++++++++++++++++++++\n 1 file changed, 36 insertions(+)\n\ndiff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh\nindex 809c662124..3521efe5d7 100644\n--- a/t/test-lib-functions.sh\n+++ b/t/test-lib-functions.sh\n@@ -1996,6 +1996,42 @@ test_trace2_data () {\n \tgrep -e '\"category\":\"'\"$1\"'\",\"key\":\"'\"$2\"'\",\"value\":\"'\"$3\"'\"'\n }\n \n+# Check that the given trace2 data event has the expected value and\n+# appears exactly once. Produces a diagnostic on failure.\n+#\n+#\ttest_trace2_data_singular [