{"thread":{"id":"66594","subject":"[PATCH] negotiator/skipping.c: parse parents before queueing","startedAt":"2026-10-19T20:17:07Z","lastAt":"2026-10-26T23:49:40Z","messageCount":15,"participants":["Taylor Blau","Patrick Steinhardt","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"555459","messageId":"dc3dfa3647c42da9170bf3289f4e8a4a8a55c8e2.1792441003.git.me@ttaylorr.com","threadId":"66594","inReplyTo":null,"subject":"[PATCH] negotiator/skipping.c: parse parents before queueing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-19T20:17:07Z","receivedAt":"2026-10-19T20:17:07Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"From: Taylor Blau <ttaylorr@openai.com>\n\nThe skipping negotiator sorts its queue by commit date, but\n`push_parent()` can add a parent before parsing it. Its date is still\nzero, so old ref tips go ahead of it.\n\nFor example, suppose a topic branch is one commit ahead of master.\nSkipping offers the topic tip and skips master's tip. Master's parent\nthen waits behind old ref tips. With enough old refs, negotiation can\nend before we offer that recent common ancestor.\n\nParse each unseen parent before adding it to the queue, so the queue\nuses its actual date. If parsing fails, leave the parent out of the\nqueue.\n\nSigned-off-by: Taylor Blau <ttaylorr@openai.com>\n---\n negotiator/skipping.c                |  5 ++++-\n t/t5552-skipping-fetch-negotiator.sh | 33 ++++++++++++++++++++++++++++\n 2 files changed, 37 insertions(+), 1 deletion(-)\n\ndiff --git a/negotiator/skipping.c b/negotiator/skipping.c\nindex db90fa77b53..ed6e88b59b4 100644\n--- a/negotiator/skipping.c\n+++ b/negotiator/skipping.c\n@@ -125,7 +125,7 @@ static void mark_common(struct data *data, struct commit *seen_commit)\n  * entry has the correct flags and ttl.\n  *\n  * This function returns 1 if an entry was found or created, and 0 otherwise\n- * (because the entry for this commit had already been popped).\n+ * (because the commit could not be parsed or its entry had already been popped).\n  */\n static int push_parent(struct data *data, struct entry *entry,\n \t\t       struct commit *to_push)\n@@ -151,6 +151,9 @@ static int push_parent(struct data *data, struct entry *entry,\n parent_found:\n \t\t;\n \t} else {\n+\t\t/* Parse the parent before using its date to order the queue. */\n+\t\tif (repo_parse_commit(the_repository, to_push))\n+\t\t\treturn 0;\n \t\tparent_entry = rev_list_push(data, to_push, 0);\n \t}\n \ndiff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh\nindex 96e224168eb..7a0b9b32474 100755\n--- a/t/t5552-skipping-fetch-negotiator.sh\n+++ b/t/t5552-skipping-fetch-negotiator.sh\n@@ -223,4 +223,37 @@ test_expect_success 'do not send \"have\" with ancestors of commits that server AC\n \thave_sent b2.c1 b2.c0\n '\n \n+test_expect_success 'recent ancestors are sent before old ref tips' '\n+\trm -rf server client trace &&\n+\tgit init server &&\n+\ttest_tick=2000000000 &&\n+\ttest_commit -C server --no-tag base &&\n+\ttest_commit -C server --no-tag parent &&\n+\ttest_commit -C server --no-tag main &&\n+\tgit clone server client &&\n+\ttest_commit -C server --no-tag to_fetch &&\n+\n+\tgit -C client checkout -b topic &&\n+\ttest_commit -C client --no-tag topic &&\n+\n+\t# These old tips have known dates, but the recent parent has no ref.\n+\t# Without parsing parents before queueing them, its date is zero and\n+\t# all of these tips are sent first, delaying it past the first batch.\n+\tfor i in $(test_seq 32)\n+\tdo\n+\t\told=$(echo \"old $i\" |\n+\t\t\tGIT_AUTHOR_DATE=\"1000000000 +0000\" \\\n+\t\t\tGIT_COMMITTER_DATE=\"1000000000 +0000\" \\\n+\t\t\tgit -C client commit-tree HEAD^{tree}) &&\n+\t\tgit -C client update-ref refs/heads/old-$i \"$old\" || return 1\n+\tdone &&\n+\n+\ttest_config -C client fetch.negotiationalgorithm skipping &&\n+\ttrace_fetch client origin &&\n+\tgit -C client rev-parse HEAD HEAD~2 >expect &&\n+\tsed -n \"s/.*fetch> have //p\" trace >haves &&\n+\tsed -n \"1,2p\" haves >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\n\nbase-commit: c51408889c3b372814bc20b9431c3b74d0d268eb\n-- \n2.56.0.187.gdc3dfa3647c\n\n"},{"id":"555676","messageId":"athgd6Ya12zRdIR_@pks.im","threadId":"66594","inReplyTo":"dc3dfa3647c42da9170bf3289f4e8a4a8a55c8e2.1792441003.git.me@ttaylorr.com","subject":"Re: [PATCH] negotiator/skipping.c: parse parents before queueing","fromName":"Patrick Steinhardt","fromEmail":"ps@pks.im","sentAt":"2026-10-21T06:49:27Z","receivedAt":"2026-10-21T06:49:27Z","isPatch":true,"sender":{"key":"ps@pks.im","avatar":"https://avatars.githubusercontent.com/u/4056630?v=4"},"body":"On Mon, Oct 19, 2026 at 03:17:07PM -0500, Taylor Blau wrote:\n> diff --git a/negotiator/skipping.c b/negotiator/skipping.c\n> index db90fa77b53..ed6e88b59b4 100644\n> --- a/negotiator/skipping.c\n> +++ b/negotiator/skipping.c\n> @@ -151,6 +151,9 @@ static int push_parent(struct data *data, struct entry *entry,\n>  parent_found:\n>  \t\t;\n>  \t} else {\n> +\t\t/* Parse the parent before using its date to order the queue. */\n> +\t\tif (repo_parse_commit(the_repository, to_push))\n> +\t\t\treturn 0;\n>  \t\tparent_entry = rev_list_push(data, to_push, 0);\n>  \t}\n\nWhy do you parse the commit in `push_parent()` instead of parsing it in\n`rev_list_push()`? The latter would also cover the other two callsites\nthat call it, and to me it's not clear why they wouldn't want to parse\nthe commits before adding it to the queue.\n\nDoing so in `rev_list_push()` would also bring it in line with the\ndefault negotiator.\n\nThere's also another call to `repo_parse_commit()` in `get_rev()`. Once\nwe start parsing the commit in `rev_list_push()` we could drop that call\nas we'd know that every commit in the queue is parsed already.\n\n> diff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh\n> index 96e224168eb..7a0b9b32474 100755\n> --- a/t/t5552-skipping-fetch-negotiator.sh\n> +++ b/t/t5552-skipping-fetch-negotiator.sh\n> @@ -223,4 +223,37 @@ test_expect_success 'do not send \"have\" with ancestors of commits that server AC\n>  \thave_sent b2.c1 b2.c0\n>  '\n>  \n> +test_expect_success 'recent ancestors are sent before old ref tips' '\n> +\trm -rf server client trace &&\n> +\tgit init server &&\n> +\ttest_tick=2000000000 &&\n> +\ttest_commit -C server --no-tag base &&\n> +\ttest_commit -C server --no-tag parent &&\n> +\ttest_commit -C server --no-tag main &&\n> +\tgit clone server client &&\n> +\ttest_commit -C server --no-tag to_fetch &&\n> +\n> +\tgit -C client checkout -b topic &&\n> +\ttest_commit -C client --no-tag topic &&\n> +\n> +\t# These old tips have known dates, but the recent parent has no ref.\n> +\t# Without parsing parents before queueing them, its date is zero and\n> +\t# all of these tips are sent first, delaying it past the first batch.\n> +\tfor i in $(test_seq 32)\n> +\tdo\n> +\t\told=$(echo \"old $i\" |\n> +\t\t\tGIT_AUTHOR_DATE=\"1000000000 +0000\" \\\n> +\t\t\tGIT_COMMITTER_DATE=\"1000000000 +0000\" \\\n> +\t\t\tgit -C client commit-tree HEAD^{tree}) &&\n> +\t\tgit -C client update-ref refs/heads/old-$i \"$old\" || return 1\n> +\tdone &&\n> +\n> +\ttest_config -C client fetch.negotiationalgorithm skipping &&\n> +\ttrace_fetch client origin &&\n> +\tgit -C client rev-parse HEAD HEAD~2 >expect &&\n> +\tsed -n \"s/.*fetch> have //p\" trace >haves &&\n> +\tsed -n \"1,2p\" haves >actual &&\n\nNit: `head -n2 haves >actual`\n\nThanks!\n\nPatrick\n\n"},{"id":"555781","messageId":"xmqqv76unagi.fsf@gitster.g","threadId":"66594","inReplyTo":"athgd6Ya12zRdIR_@pks.im","subject":"Re: [PATCH] negotiator/skipping.c: parse parents before queueing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-21T19:02:37Z","receivedAt":"2026-10-21T19:02:37Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Patrick Steinhardt <ps@pks.im> writes:\n\n> On Mon, Oct 19, 2026 at 03:17:07PM -0500, Taylor Blau wrote:\n>> diff --git a/negotiator/skipping.c b/negotiator/skipping.c\n>> index db90fa77b53..ed6e88b59b4 100644\n>> --- a/negotiator/skipping.c\n>> +++ b/negotiator/skipping.c\n>> @@ -151,6 +151,9 @@ static int push_parent(struct data *data, struct entry *entry,\n>>  parent_found:\n>>  \t\t;\n>>  \t} else {\n>> +\t\t/* Parse the parent before using its date to order the queue. */\n>> +\t\tif (repo_parse_commit(the_repository, to_push))\n>> +\t\t\treturn 0;\n>>  \t\tparent_entry = rev_list_push(data, to_push, 0);\n>>  \t}\n>\n> Why do you parse the commit in `push_parent()` instead of parsing it in\n> `rev_list_push()`? The latter would also cover the other two callsites\n> that call it, and to me it's not clear why they wouldn't want to parse\n> the commits before adding it to the queue.\n>\n> Doing so in `rev_list_push()` would also bring it in line with the\n> default negotiator.\n>\n> There's also another call to `repo_parse_commit()` in `get_rev()`. Once\n> we start parsing the commit in `rev_list_push()` we could drop that call\n> as we'd know that every commit in the queue is parsed already.\n\nThanks for noticing this.\n\n"},{"id":"555903","messageId":"atp0Q5dFHQHAZAov@com-79390","threadId":"66594","inReplyTo":"athgd6Ya12zRdIR_@pks.im","subject":"Re: [PATCH] negotiator/skipping.c: parse parents before queueing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-22T20:38:27Z","receivedAt":"2026-10-22T20:38:27Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Wed, Oct 21, 2026 at 08:49:27AM +0200, Patrick Steinhardt wrote:\n> On Mon, Oct 19, 2026 at 03:17:07PM -0500, Taylor Blau wrote:\n> > diff --git a/negotiator/skipping.c b/negotiator/skipping.c\n> > index db90fa77b53..ed6e88b59b4 100644\n> > --- a/negotiator/skipping.c\n> > +++ b/negotiator/skipping.c\n> > @@ -151,6 +151,9 @@ static int push_parent(struct data *data, struct entry *entry,\n> >  parent_found:\n> >  \t\t;\n> >  \t} else {\n> > +\t\t/* Parse the parent before using its date to order the queue. */\n> > +\t\tif (repo_parse_commit(the_repository, to_push))\n> > +\t\t\treturn 0;\n> >  \t\tparent_entry = rev_list_push(data, to_push, 0);\n> >  \t}\n>\n> Why do you parse the commit in `push_parent()` instead of parsing it in\n> `rev_list_push()`? The latter would also cover the other two callsites\n> that call it, and to me it's not clear why they wouldn't want to parse\n> the commits before adding it to the queue.\n>\n> Doing so in `rev_list_push()` would also bring it in line with the\n> default negotiator.\n\nI originally wrote it this way because the skipping negotiator's version\nof `rev_list_push()` returns a pointer to an 'entry' struct, whereas the\ndefault negotiator's simply returns void. But the only caller that needs\nto learn to handle a NULL return value from that function is simply\n`push_parent()` itself, since the other two callers already disregard\nthe return value.\n\nI agree that performing the parse in `rev_list_push()` makes more sense\nfrom both an ergonomic and consistency perspective, so I have moved it\nthere for the next round.\n\n> There's also another call to `repo_parse_commit()` in `get_rev()`. Once\n> we start parsing the commit in `rev_list_push()` we could drop that call\n> as we'd know that every commit in the queue is parsed already.\n\nThat makes sense. The commit struct comes from the queue, and we know\nthat the queue always has fully-parsed commits, so there is no need to\nre-parse it (even though the second and subsequent calls are noops).\nI've removed it.\n\n> > +\tsed -n \"s/.*fetch> have //p\" trace >haves &&\n> > +\tsed -n \"1,2p\" haves >actual &&\n>\n> Nit: `head -n2 haves >actual`\n\nThanks, I realized when responding here that we don't even need the\n'haves' intermediate file, so we can replace this and the above line\nwith:\n\n    sed -n \"s/.*fetch> have //p\" trace | head -n 2 >actual &&\n\nThanks for reviewing!\n\nThanks,\nTaylor\n\n"},{"id":"555918","messageId":"9e9ae442333b5e2ccc0905c44c2c3e16c49481a7.1792704069.git.me@ttaylorr.com","threadId":"66594","inReplyTo":"dc3dfa3647c42da9170bf3289f4e8a4a8a55c8e2.1792441003.git.me@ttaylorr.com","subject":"[PATCH v2] negotiator/skipping.c: parse parents before queueing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-22T21:21:13Z","receivedAt":"2026-10-22T21:21:13Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"From: Taylor Blau <ttaylorr@openai.com>\n\nThe skipping negotiator sorts its queue by commit date, but callers\nwhich push commits onto that queue, such as `push_parent()`, can add a\nparent before parsing it. When an unparsed commit enters the queue, its\ndate is still zero, so old ref tips may sort ahead of it.\n\nFor example, suppose a topic branch is one commit ahead of master.\nSkipping offers the topic tip and skips master's tip. Master's parent\nthen waits behind old ref tips. With enough old refs, negotiation can\nend before we offer that recent common ancestor.\n\nParse each unseen parent before adding it to the queue, so the queue\nuses its actual date. If parsing fails, leave the parent out of the\nqueue.\n\nSince the sole caller of `prio_queue_put()` ensures that commits are\nfully parsed before pushing them onto the queue, we can avoid a noop\ncall to `repo_parse_commit()` within `get_rev()`.\n\nNote that unlike the default negotiator, the skipping negotiator's\nversion of `rev_list_push()` returns a pointer to an 'entry' struct. As\na consequence, callers must be prepared to handle a NULL return value.\nThe caller in `push_parent()` gains a NULL-ness check, and the remaining\ntwo callers continue to ignore the return value from `rev_list_push()`.\n\nSigned-off-by: Taylor Blau <ttaylorr@openai.com>\n---\n negotiator/skipping.c                |  8 +++++--\n t/t5552-skipping-fetch-negotiator.sh | 32 ++++++++++++++++++++++++++++\n 2 files changed, 38 insertions(+), 2 deletions(-)\n\ndiff --git a/negotiator/skipping.c b/negotiator/skipping.c\nindex db90fa77b53..0ae5f28d353 100644\n--- a/negotiator/skipping.c\n+++ b/negotiator/skipping.c\n@@ -66,6 +66,9 @@ static struct entry *rev_list_push(struct data *data, struct commit *commit, int\n \tstruct entry *entry;\n \tcommit->object.flags |= mark | SEEN;\n \n+\tif (repo_parse_commit(the_repository, commit))\n+\t\treturn NULL;\n+\n \tCALLOC_ARRAY(entry, 1);\n \tentry->commit = commit;\n \tprio_queue_put(&data->rev_list, entry);\n@@ -125,7 +128,7 @@ static void mark_common(struct data *data, struct commit *seen_commit)\n  * entry has the correct flags and ttl.\n  *\n  * This function returns 1 if an entry was found or created, and 0 otherwise\n- * (because the entry for this commit had already been popped).\n+ * (because the commit could not be parsed or its entry had already been popped).\n  */\n static int push_parent(struct data *data, struct entry *entry,\n \t\t       struct commit *to_push)\n@@ -152,6 +155,8 @@ static int push_parent(struct data *data, struct entry *entry,\n \t\t;\n \t} else {\n \t\tparent_entry = rev_list_push(data, to_push, 0);\n+\t\tif (!parent_entry)\n+\t\t\treturn 0;\n \t}\n \n \tif (entry->commit->object.flags & (COMMON | ADVERTISED)) {\n@@ -194,7 +199,6 @@ static const struct object_id *get_rev(struct data *data)\n \t\tif (!(commit->object.flags & COMMON) && !entry->ttl)\n \t\t\tto_send = commit;\n \n-\t\trepo_parse_commit(the_repository, commit);\n \t\tfor (p = commit->parents; p; p = p->next)\n \t\t\tparent_pushed |= push_parent(data, entry, p->item);\n \ndiff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh\nindex 96e224168eb..7479424ca94 100755\n--- a/t/t5552-skipping-fetch-negotiator.sh\n+++ b/t/t5552-skipping-fetch-negotiator.sh\n@@ -223,4 +223,36 @@ test_expect_success 'do not send \"have\" with ancestors of commits that server AC\n \thave_sent b2.c1 b2.c0\n '\n \n+test_expect_success 'recent ancestors are sent before old ref tips' '\n+\trm -rf server client trace &&\n+\tgit init server &&\n+\ttest_tick=2000000000 &&\n+\ttest_commit -C server --no-tag base &&\n+\ttest_commit -C server --no-tag parent &&\n+\ttest_commit -C server --no-tag main &&\n+\tgit clone server client &&\n+\ttest_commit -C server --no-tag to_fetch &&\n+\n+\tgit -C client checkout -b topic &&\n+\ttest_commit -C client --no-tag topic &&\n+\n+\t# These old tips have known dates, but the recent parent has no ref.\n+\t# Without parsing parents before queueing them, its date is zero and\n+\t# all of these tips are sent first, delaying it past the first batch.\n+\tfor i in $(test_seq 32)\n+\tdo\n+\t\told=$(echo \"old $i\" |\n+\t\t\tGIT_AUTHOR_DATE=\"1000000000 +0000\" \\\n+\t\t\tGIT_COMMITTER_DATE=\"1000000000 +0000\" \\\n+\t\t\tgit -C client commit-tree HEAD^{tree}) &&\n+\t\tgit -C client update-ref refs/heads/old-$i \"$old\" || return 1\n+\tdone &&\n+\n+\ttest_config -C client fetch.negotiationalgorithm skipping &&\n+\ttrace_fetch client origin &&\n+\tgit -C client rev-parse HEAD HEAD~2 >expect &&\n+\tsed -n \"s/.*fetch> have //p\" trace | head -n 2 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_done\n\nRange-diff against v1:\n1:  cfb16d16d8c ! 1:  9e9ae442333 negotiator/skipping.c: parse parents before queueing\n    @@ Metadata\n      ## Commit message ##\n         negotiator/skipping.c: parse parents before queueing\n     \n    -    The skipping negotiator sorts its queue by commit date, but\n    -    `push_parent()` can add a parent before parsing it. Its date is still\n    -    zero, so old ref tips go ahead of it.\n    +    The skipping negotiator sorts its queue by commit date, but callers\n    +    which push commits onto that queue, such as `push_parent()`, can add a\n    +    parent before parsing it. When an unparsed commit enters the queue, its\n    +    date is still zero, so old ref tips may sort ahead of it.\n     \n         For example, suppose a topic branch is one commit ahead of master.\n         Skipping offers the topic tip and skips master's tip. Master's parent\n    @@ Commit message\n         uses its actual date. If parsing fails, leave the parent out of the\n         queue.\n     \n    +    Since the sole caller of `prio_queue_put()` ensures that commits are\n    +    fully parsed before pushing them onto the queue, we can avoid a noop\n    +    call to `repo_parse_commit()` within `get_rev()`.\n    +\n    +    Note that unlike the default negotiator, the skipping negotiator's\n    +    version of `rev_list_push()` returns a pointer to an 'entry' struct. As\n    +    a consequence, callers must be prepared to handle a NULL return value.\n    +    The caller in `push_parent()` gains a NULL-ness check, and the remaining\n    +    two callers continue to ignore the return value from `rev_list_push()`.\n    +\n         Signed-off-by: Taylor Blau <ttaylorr@openai.com>\n    -    Signed-off-by: Taylor Blau <me@ttaylorr.com>\n     \n      ## negotiator/skipping.c ##\n    +@@ negotiator/skipping.c: static struct entry *rev_list_push(struct data *data, struct commit *commit, int\n    + \tstruct entry *entry;\n    + \tcommit->object.flags |= mark | SEEN;\n    + \n    ++\tif (repo_parse_commit(the_repository, commit))\n    ++\t\treturn NULL;\n    ++\n    + \tCALLOC_ARRAY(entry, 1);\n    + \tentry->commit = commit;\n    + \tprio_queue_put(&data->rev_list, entry);\n     @@ negotiator/skipping.c: static void mark_common(struct data *data, struct commit *seen_commit)\n       * entry has the correct flags and ttl.\n       *\n    @@ negotiator/skipping.c: static void mark_common(struct data *data, struct commit\n      static int push_parent(struct data *data, struct entry *entry,\n      \t\t       struct commit *to_push)\n     @@ negotiator/skipping.c: static int push_parent(struct data *data, struct entry *entry,\n    - parent_found:\n      \t\t;\n      \t} else {\n    -+\t\t/* Parse the parent before using its date to order the queue. */\n    -+\t\tif (repo_parse_commit(the_repository, to_push))\n    -+\t\t\treturn 0;\n      \t\tparent_entry = rev_list_push(data, to_push, 0);\n    ++\t\tif (!parent_entry)\n    ++\t\t\treturn 0;\n      \t}\n      \n    + \tif (entry->commit->object.flags & (COMMON | ADVERTISED)) {\n    +@@ negotiator/skipping.c: static const struct object_id *get_rev(struct data *data)\n    + \t\tif (!(commit->object.flags & COMMON) && !entry->ttl)\n    + \t\t\tto_send = commit;\n    + \n    +-\t\trepo_parse_commit(the_repository, commit);\n    + \t\tfor (p = commit->parents; p; p = p->next)\n    + \t\t\tparent_pushed |= push_parent(data, entry, p->item);\n    + \n     \n      ## t/t5552-skipping-fetch-negotiator.sh ##\n     @@ t/t5552-skipping-fetch-negotiator.sh: test_expect_success 'do not send \"have\" with ancestors of commits that server AC\n    @@ t/t5552-skipping-fetch-negotiator.sh: test_expect_success 'do not send \"have\" wi\n     +\ttest_config -C client fetch.negotiationalgorithm skipping &&\n     +\ttrace_fetch client origin &&\n     +\tgit -C client rev-parse HEAD HEAD~2 >expect &&\n    -+\tsed -n \"s/.*fetch> have //p\" trace >haves &&\n    -+\tsed -n \"1,2p\" haves >actual &&\n    ++\tsed -n \"s/.*fetch> have //p\" trace | head -n 2 >actual &&\n     +\ttest_cmp expect actual\n     +'\n     +\n\nbase-commit: c51408889c3b372814bc20b9431c3b74d0d268eb\n-- \n2.56.0.187.g9e9ae442333\n\n"},{"id":"555926","messageId":"xmqq1p9he7cr.fsf@gitster.g","threadId":"66594","inReplyTo":"atp0Q5dFHQHAZAov@com-79390","subject":"Re: [PATCH] negotiator/skipping.c: parse parents before queueing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-22T21:46:44Z","receivedAt":"2026-10-22T21:46:44Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau <ttaylorr@openai.com> writes:\n\n> Thanks, I realized when responding here that we don't even need the\n> 'haves' intermediate file, so we can replace this and the above line\n> with:\n>\n>     sed -n \"s/.*fetch> have //p\" trace | head -n 2 >actual &&\n\nLooks good enough.  Thanks.\n\nI wondered if we can avoid pipes, but the best I could come up with\nwas this:\n\n    sed -n '/.*fetch> have /{s///p;x;/./q;H;}' trace >actual\n\nwhich is not exactly readable.  Even folks on process/fork starved\nplatforms would not appreciate it X-<.\n\n\n"},{"id":"555930","messageId":"xmqqld7pcqku.fsf@gitster.g","threadId":"66594","inReplyTo":"9e9ae442333b5e2ccc0905c44c2c3e16c49481a7.1792704069.git.me@ttaylorr.com","subject":"Re: [PATCH v2] negotiator/skipping.c: parse parents before queueing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-22T22:34:25Z","receivedAt":"2026-10-22T22:34:25Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau <ttaylorr@openai.com> writes:\n\n> From: Taylor Blau <ttaylorr@openai.com>\n>\n> The skipping negotiator sorts its queue by commit date, but callers\n> which push commits onto that queue, such as `push_parent()`, can add a\n> parent before parsing it. When an unparsed commit enters the queue, its\n> date is still zero, so old ref tips may sort ahead of it.\n>\n> For example, suppose a topic branch is one commit ahead of master.\n> Skipping offers the topic tip and skips master's tip. Master's parent\n> then waits behind old ref tips. With enough old refs, negotiation can\n> end before we offer that recent common ancestor.\n>\n> Parse each unseen parent before adding it to the queue, so the queue\n> uses its actual date. If parsing fails, leave the parent out of the\n> queue.\n>\n> Since the sole caller of `prio_queue_put()` ensures that commits are\n> fully parsed before pushing them onto the queue, we can avoid a noop\n> call to `repo_parse_commit()` within `get_rev()`.\n>\n> Note that unlike the default negotiator, the skipping negotiator's\n> version of `rev_list_push()` returns a pointer to an 'entry' struct. As\n> a consequence, callers must be prepared to handle a NULL return value.\n> The caller in `push_parent()` gains a NULL-ness check, and the remaining\n> two callers continue to ignore the return value from `rev_list_push()`.\n>\n> Signed-off-by: Taylor Blau <ttaylorr@openai.com>\n> ---\n>  negotiator/skipping.c                |  8 +++++--\n>  t/t5552-skipping-fetch-negotiator.sh | 32 ++++++++++++++++++++++++++++\n>  2 files changed, 38 insertions(+), 2 deletions(-)\n>\n> diff --git a/negotiator/skipping.c b/negotiator/skipping.c\n> index db90fa77b53..0ae5f28d353 100644\n> --- a/negotiator/skipping.c\n> +++ b/negotiator/skipping.c\n> @@ -66,6 +66,9 @@ static struct entry *rev_list_push(struct data *data, struct commit *commit, int\n>  \tstruct entry *entry;\n>  \tcommit->object.flags |= mark | SEEN;\n>  \n> +\tif (repo_parse_commit(the_repository, commit))\n> +\t\treturn NULL;\n> +\n>  \tCALLOC_ARRAY(entry, 1);\n>  \tentry->commit = commit;\n>  \tprio_queue_put(&data->rev_list, entry);\n> @@ -125,7 +128,7 @@ static void mark_common(struct data *data, struct commit *seen_commit)\n>   * entry has the correct flags and ttl.\n>   *\n>   * This function returns 1 if an entry was found or created, and 0 otherwise\n> - * (because the entry for this commit had already been popped).\n> + * (because the commit could not be parsed or its entry had already been popped).\n>   */\n>  static int push_parent(struct data *data, struct entry *entry,\n>  \t\t       struct commit *to_push)\n> @@ -152,6 +155,8 @@ static int push_parent(struct data *data, struct entry *entry,\n>  \t\t;\n>  \t} else {\n>  \t\tparent_entry = rev_list_push(data, to_push, 0);\n> +\t\tif (!parent_entry)\n> +\t\t\treturn 0;\n>  \t}\n>  \n>  \tif (entry->commit->object.flags & (COMMON | ADVERTISED)) {\n> @@ -194,7 +199,6 @@ static const struct object_id *get_rev(struct data *data)\n>  \t\tif (!(commit->object.flags & COMMON) && !entry->ttl)\n>  \t\t\tto_send = commit;\n>  \n> -\t\trepo_parse_commit(the_repository, commit);\n>  \t\tfor (p = commit->parents; p; p = p->next)\n>  \t\t\tparent_pushed |= push_parent(data, entry, p->item);\n\nMoving the parsing to rev_list_push() means that when a commit\ncannot be parsed, it is marked as 'SEEN' and NULL is returned.  The\ncaller, push_parent(), notices this NULL return and naturally\nreturns without pushing the parent.  However, when the same\nunparsable commit is encountered again (perhaps as a parent of\nanother commit), push_parent() sees that it is already 'SEEN' and\nlooks for it in the priority queue.  It will fail to find it and\ntrigger BUG(\"missing parent in priority queue\").\n\nPerhaps rev_list_push() should mark the commit as 'SEEN' only after\nrepo_parse_commit() succeeds?\n\nWhile this issue only arises with a corrupt repository or when a\npromisor remote goes offline, a failure to parse should not trigger\na BUG(), which should be reserved for programming errors.\n\nThanks.\n\n"},{"id":"555931","messageId":"atqSs0yzXw0cf3cH@com-79390","threadId":"66594","inReplyTo":"xmqqld7pcqku.fsf@gitster.g","subject":"Re: [PATCH v2] negotiator/skipping.c: parse parents before queueing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-22T22:48:19Z","receivedAt":"2026-10-22T22:48:19Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Oct 22, 2026 at 03:34:25PM -0700, Junio C Hamano wrote:\n> Perhaps rev_list_push() should mark the commit as 'SEEN' only after\n> repo_parse_commit() succeeds?\n\nI considered this but ultimately decided against it, since the default\nnegotiator does:\n\n    static void rev_list_push(struct negotiation_state *ns,\n                              struct commit *commit, int mark)\n    {\n        if (!(commit->object.flags & mark)) {\n            commit->object.flags |= mark;\n\n            if (repo_parse_commit(the_repository, commit))\n                return;\n\n    /* ... */\n\nThere are various callers of rev_list_push() in the default negotiator,\nbut all include SEEN in the provided 'mark'.\n\nThanks,\nTaylor\n\n"},{"id":"555932","messageId":"atqS-JB5ayn_ixmG@com-79390","threadId":"66594","inReplyTo":"xmqq1p9he7cr.fsf@gitster.g","subject":"Re: [PATCH] negotiator/skipping.c: parse parents before queueing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-22T22:49:28Z","receivedAt":"2026-10-22T22:49:28Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Thu, Oct 22, 2026 at 02:46:44PM -0700, Junio C Hamano wrote:\n> I wondered if we can avoid pipes, but the best I could come up with\n> was this:\n>\n>     sed -n '/.*fetch> have /{s///p;x;/./q;H;}' trace >actual\n\nYeah, I wondered something similar, but figured that writing it out\nwould be fairly inscrutable. Seems like it was ;-).\n\nThanks,\nTaylor\n\n"},{"id":"556046","messageId":"xmqqh5icbial.fsf@gitster.g","threadId":"66594","inReplyTo":"atqSs0yzXw0cf3cH@com-79390","subject":"Re: [PATCH v2] negotiator/skipping.c: parse parents before queueing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-23T14:30:58Z","receivedAt":"2026-10-23T14:30:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau <ttaylorr@openai.com> writes:\n\n> On Thu, Oct 22, 2026 at 03:34:25PM -0700, Junio C Hamano wrote:\n>> Perhaps rev_list_push() should mark the commit as 'SEEN' only after\n>> repo_parse_commit() succeeds?\n>\n> I considered this but ultimately decided against it, since the default\n> negotiator does:\n> ...\n> There are various callers of rev_list_push() in the default negotiator,\n> but all include SEEN in the provided 'mark'.\n\nIt is understandable that we do not want to widen the scope of the\npatch by adding a preparatory \"fix the existing ones\" step, but I do\nnot think \"the existing ones make the same mistake\" is particularly\na good excuse to add new one.  Would a hybrid approach to (1) avoid\ncopying a mistake to new code and (2) document near the new code and/or\nin the log message that the difference is deliberate and fixing existing\nones are #leftoverbits and not addressed in this commit?\n\n\n"},{"id":"556129","messageId":"atv1uK2ZIupPgrES@com-79390","threadId":"66594","inReplyTo":"xmqqh5icbial.fsf@gitster.g","subject":"Re: [PATCH v2] negotiator/skipping.c: parse parents before queueing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-24T00:03:04Z","receivedAt":"2026-10-24T00:03:04Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"On Fri, Oct 23, 2026 at 07:30:58AM -0700, Junio C Hamano wrote:\n> Taylor Blau <ttaylorr@openai.com> writes:\n>\n> > On Thu, Oct 22, 2026 at 03:34:25PM -0700, Junio C Hamano wrote:\n> >> Perhaps rev_list_push() should mark the commit as 'SEEN' only after\n> >> repo_parse_commit() succeeds?\n> >\n> > I considered this but ultimately decided against it, since the default\n> > negotiator does:\n> > ...\n> > There are various callers of rev_list_push() in the default negotiator,\n> > but all include SEEN in the provided 'mark'.\n>\n> It is understandable that we do not want to widen the scope of the\n> patch by adding a preparatory \"fix the existing ones\" step, but I do\n> not think \"the existing ones make the same mistake\" is particularly\n> a good excuse to add new one.\n\nI think that's fair. For what it's worth, when writing it my thought\nprocess was \"let's be as consistent as possible with the most\nbattle-tested negotiator\" as opposed to \"this is broken, let's repeat\nthe breakage elsewhere\".\n\nBut looking at it more closely, I think you're right that there is a bug\nhere in the default negotiator. So...\n\n> Would a hybrid approach to (1) avoid copying a mistake to new code and\n> (2) document near the new code and/or in the log message that the\n> difference is deliberate and fixing existing ones are #leftoverbits\n> and not addressed in this commit?\n\nI think a hybrid approach is fine, though I am happy to fix the default\nnegotiator alongside this series. I have a patch for it locally, but I'm\nhappy to send either depending on your preference.\n\nThanks,\nTaylor\n\n"},{"id":"556135","messageId":"xmqqqzhf26ow.fsf@gitster.g","threadId":"66594","inReplyTo":"atv1uK2ZIupPgrES@com-79390","subject":"Re: [PATCH v2] negotiator/skipping.c: parse parents before queueing","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2026-10-24T02:06:23Z","receivedAt":"2026-10-24T02:06:23Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Taylor Blau <ttaylorr@openai.com> writes:\n\n> But looking at it more closely, I think you're right that there is a bug\n> here in the default negotiator. So...\n>\n>> Would a hybrid approach to (1) avoid copying a mistake to new code and\n>> (2) document near the new code and/or in the log message that the\n>> difference is deliberate and fixing existing ones are #leftoverbits\n>> and not addressed in this commit?\n>\n> I think a hybrid approach is fine, though I am happy to fix the default\n> negotiator alongside this series. I have a patch for it locally, but I'm\n> happy to send either depending on your preference.\n\nIf you have a clear grasp of the existing bug and the fix is\nstraightforward, let's do the fix as part of this series.\n\nThanks.\n\n"},{"id":"556415","messageId":"cover.1793058562.git.ttaylorr@openai.com","threadId":"66594","inReplyTo":"dc3dfa3647c42da9170bf3289f4e8a4a8a55c8e2.1792441003.git.me@ttaylorr.com","subject":"[PATCH v3 0/2] negotiator: parse commits before queueing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-26T23:49:28Z","receivedAt":"2026-10-26T23:49:28Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"This is a reroll of my patches to fix a pair of bugs in the default and\nskipping negotiators.\n\nThis version fixes the SEEN bookkeeping issue Junio pointed out in v2,\nand adds a separate fix for the same mistake in the default negotiator.\n\nThe original problem is that the skipping negotiator can queue unparsed\nparents with a date of zero. Old ref tips then sort ahead of recent\nancestors, so negotiation can end before we offer a recent common\nancestor. As in v2, the fix parses commits in rev_list_push(), where\ntheir dates are needed to order the queue, and removes the later parse\nfrom get_rev().\n\nBut v2 set SEEN before attempting that parse. If parsing failed, another\nchild sharing the missing parent would look for a queue entry that did\nnot exist and trigger BUG(\"missing parent in priority queue\"). Both\nrev_list_push() implementations set SEEN only after parsing succeeds:\n\n  - The first patch fixes the existing bug in the default negotiator.\n    Marking a missing ancestor SEEN can cause mark_common() to decrement\n    non_common_revs for a commit that was never queued, ending\n    negotiation while other commits remain.\n\n  - The second patch is an updated version of the previous round which\n    applies the same fix to the skipping negotiator. Relative to the\n    earlier round, it also adds a test where two tips share a missing\n    parent and fetching retrieves that parent without hitting the BUG().\n\nThanks to Junio and Patrick for their review. The range-diff against v2\nis below.\n\nThanks in advance for your review!\n\nTaylor Blau (2):\n  negotiator/default.c: mark commits seen only after parsing\n  negotiator/skipping.c: parse parents before queueing\n\n negotiator/default.c                 |  3 +-\n negotiator/skipping.c                |  9 +++-\n t/t5552-skipping-fetch-negotiator.sh | 63 ++++++++++++++++++++++++++++\n t/t5702-protocol-v2.sh               | 31 ++++++++++++++\n 4 files changed, 102 insertions(+), 4 deletions(-)\n\nRange-diff against v2:\n-:  ----------- > 1:  5a6af916635 negotiator/default.c: mark commits seen only after parsing\n1:  c1674625a65 ! 2:  7365a2ab180 negotiator/skipping.c: parse parents before queueing\n    @@ Commit message\n         then waits behind old ref tips. With enough old refs, negotiation can\n         end before we offer that recent common ancestor.\n     \n    -    Parse each unseen parent before adding it to the queue, so the queue\n    -    uses its actual date. If parsing fails, leave the parent out of the\n    -    queue.\n    +    Parse commits in rev_list_push() before adding them to the queue, so\n    +    the queue uses their actual dates. If parsing fails, leave the commit\n    +    out of the queue. Move the parsing from get_rev() to the insertion\n    +    point, where the date is needed to order the queue.\n     \n    -    Since the sole caller of `prio_queue_put()` ensures that commits are\n    -    fully parsed before pushing them onto the queue, we can avoid a noop\n    -    call to `repo_parse_commit()` within `get_rev()`.\n    +    Set SEEN only after parsing succeeds, following the default negotiator\n    +    after the preceding fix. push_parent() assumes that a commit marked\n    +    SEEN but not POPPED has an entry in the queue. Marking an unparseable\n    +    parent SEEN would violate that assumption and trigger a BUG() if another\n    +    child tried to push the same parent.\n     \n    -    Note that unlike the default negotiator, the skipping negotiator's\n    -    version of `rev_list_push()` returns a pointer to an 'entry' struct. As\n    -    a consequence, callers must be prepared to handle a NULL return value.\n    -    The caller in `push_parent()` gains a NULL-ness check, and the remaining\n    -    two callers continue to ignore the return value from `rev_list_push()`.\n    +    Unlike the default negotiator, the skipping negotiator's version of\n    +    rev_list_push() returns a pointer to an entry struct. Check for NULL in\n    +    push_parent() before using that entry. The other two callers continue\n    +    to ignore the return value.\n    +\n    +    Add tests for the ordering of recent ancestors relative to old ref tips\n    +    and for two tips sharing a missing parent. In the latter case, fetching\n    +    from a server that has the parent should retrieve it without triggering\n    +    the missing-entry BUG().\n     \n         Signed-off-by: Taylor Blau <ttaylorr@openai.com>\n     \n      ## negotiator/skipping.c ##\n    -@@ negotiator/skipping.c: static struct entry *rev_list_push(struct data *data, struct commit *commit, int\n    +@@ negotiator/skipping.c: static int compare(const void *a_, const void *b_, void *data UNUSED)\n    + static struct entry *rev_list_push(struct data *data, struct commit *commit, int mark)\n    + {\n      \tstruct entry *entry;\n    - \tcommit->object.flags |= mark | SEEN;\n    - \n    ++\n     +\tif (repo_parse_commit(the_repository, commit))\n     +\t\treturn NULL;\n     +\n    + \tcommit->object.flags |= mark | SEEN;\n    + \n      \tCALLOC_ARRAY(entry, 1);\n    - \tentry->commit = commit;\n    - \tprio_queue_put(&data->rev_list, entry);\n     @@ negotiator/skipping.c: static void mark_common(struct data *data, struct commit *seen_commit)\n       * entry has the correct flags and ttl.\n       *\n    @@ t/t5552-skipping-fetch-negotiator.sh: test_expect_success 'do not send \"have\" wi\n     +\tsed -n \"s/.*fetch> have //p\" trace | head -n 2 >actual &&\n     +\ttest_cmp expect actual\n     +'\n    ++\n    ++test_expect_success 'missing parent shared by two tips is not treated as queued' '\n    ++\trm -rf server client trace &&\n    ++\tgit init server &&\n    ++\ttest_commit -C server base &&\n    ++\tparent=$(git -C server rev-parse HEAD) &&\n    ++\ttest_commit -C server to_fetch &&\n    ++\n    ++\tgit init client &&\n    ++\ttree=$(git -C client mktree </dev/null) &&\n    ++\t# Write the tips directly, bypassing the parent lookup in commit-tree.\n    ++\t# Both tips name a parent that only the server has.\n    ++\tfor i in 1 2\n    ++\tdo\n    ++\t\tcat >commit <<-EOF_COMMIT &&\n    ++\t\ttree $tree\n    ++\t\tparent $parent\n    ++\t\tauthor A U Thor <author@example.com> 1000000000 +0000\n    ++\t\tcommitter C O Mitter <committer@example.com> 1000000000 +0000\n    ++\n    ++\t\ttip $i\n    ++\t\tEOF_COMMIT\n    ++\t\ttip=$(git -C client hash-object -w -t commit --stdin <commit) &&\n    ++\t\tgit -C client update-ref refs/heads/tip-$i \"$tip\" || return 1\n    ++\tdone &&\n    ++\n    ++\ttest_config -C client fetch.negotiationalgorithm skipping &&\n    ++\ttrace_fetch client \"$(pwd)/server\" \\\n    ++\t\t--negotiation-tip=tip-1 --negotiation-tip=tip-2 &&\n    ++\tgit -C client cat-file -e \"$parent\"\n    ++'\n     +\n      test_done\n\nbase-commit: c51408889c3b372814bc20b9431c3b74d0d268eb\n-- \n2.56.0.187.g9e9ae442333\n\n"},{"id":"556416","messageId":"5a6af9166354f0c0d0c229f0ed57b495af24090c.1793058562.git.ttaylorr@openai.com","threadId":"66594","inReplyTo":"cover.1793058562.git.ttaylorr@openai.com","subject":"[PATCH v3 1/2] negotiator/default.c: mark commits seen only after parsing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-26T23:49:37Z","receivedAt":"2026-10-26T23:49:37Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"rev_list_push() sets SEEN before parsing a commit, but leaves the commit\nout of the queue if parsing fails. This makes SEEN describe an attempted\ninsertion rather than a commit that was actually queued.\n\nmark_common() relies on SEEN and POPPED to decide whether a newly common\ncommit contributes to non_common_revs. If a missing parent is marked SEEN\nand later reached while acknowledging one of its children, mark_common()\nsubtracts it even though rev_list_push() never incremented the counter.\nThis can stop negotiation while other non-common commits remain queued.\n\nSet the requested flags only after parsing succeeds. Successful\ninsertions are unchanged, and a failed parse no longer marks the commit\nas queued. This also lets the skipping negotiator follow the same order\nwhen it starts parsing commits in rev_list_push().\n\nAdd a negotiation-only fetch test that fills the first batch with commits\nabove a missing parent, leaving an older, unrelated tip queued. The ACK\nfor the first branch must not stop negotiation before the second tip is\nsent and acknowledged. Before this fix, the second tip is omitted and\nthe negotiation-only fetch fails.\n\nSigned-off-by: Taylor Blau <ttaylorr@openai.com>\n---\n negotiator/default.c   |  3 +--\n t/t5702-protocol-v2.sh | 31 +++++++++++++++++++++++++++++++\n 2 files changed, 32 insertions(+), 2 deletions(-)\n\ndiff --git a/negotiator/default.c b/negotiator/default.c\nindex 19cdf3808cf..32cba7ed8d9 100644\n--- a/negotiator/default.c\n+++ b/negotiator/default.c\n@@ -26,11 +26,10 @@ static void rev_list_push(struct negotiation_state *ns,\n \t\t\t  struct commit *commit, int mark)\n {\n \tif (!(commit->object.flags & mark)) {\n-\t\tcommit->object.flags |= mark;\n-\n \t\tif (repo_parse_commit(the_repository, commit))\n \t\t\treturn;\n \n+\t\tcommit->object.flags |= mark;\n \t\tprio_queue_put(&ns->rev_list, commit);\n \n \t\tif (!(commit->object.flags & COMMON))\ndiff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh\nindex 7e69ba7f843..b1768cbdbc5 100755\n--- a/t/t5702-protocol-v2.sh\n+++ b/t/t5702-protocol-v2.sh\n@@ -912,6 +912,37 @@ test_expect_success 'file:// --negotiate-only' '\n \ttest_grep \"$COMMON\" out\n '\n \n+test_expect_success 'negotiate-only continues after ACK with a missing ancestor' '\n+\trm -rf server client &&\n+\tgit init server &&\n+\ttest_commit -C server --no-tag b &&\n+\tgit -C server branch b &&\n+\tgit -C server checkout --orphan a &&\n+\ttest_commit -C server --no-tag missing &&\n+\tmissing=$(git -C server rev-parse HEAD) &&\n+\n+\t# Fill the first batch with this branch, leaving b for the next one.\n+\tfor i in $(test_seq 16)\n+\tdo\n+\t\ttest_commit -C server --no-tag a$i || return 1\n+\tdone &&\n+\tgit clone --no-hardlinks server client &&\n+\t# Force a parse failure in the client. Keep the parent on the\n+\t# server so it can acknowledge the branch normally.\n+\trm \"client/.git/objects/$(test_oid_to_path \"$missing\")\" &&\n+\n+\t# ACKing a must not count the missing ancestor as a queued commit\n+\t# and stop negotiation before sending b.\n+\tgit -c protocol.version=2 -c core.commitGraph=false \\\n+\t\t-c fetch.negotiationAlgorithm=consecutive -C client fetch \\\n+\t\t--negotiate-only --negotiation-tip=a --negotiation-tip=origin/b \\\n+\t\torigin >out &&\n+\tgit -C client rev-parse a origin/b >tips &&\n+\tsort tips >expect &&\n+\tsort out >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'file:// --negotiate-only with protocol v0' '\n \tSERVER=\"server\" &&\n \tURI=\"file://$(pwd)/server\" &&\n-- \n2.56.0.187.g9e9ae442333\n\n\n"},{"id":"556417","messageId":"7365a2ab180bc4b534dac3b7515a398a987e8384.1793058562.git.ttaylorr@openai.com","threadId":"66594","inReplyTo":"cover.1793058562.git.ttaylorr@openai.com","subject":"[PATCH v3 2/2] negotiator/skipping.c: parse parents before queueing","fromName":"Taylor Blau","fromEmail":"ttaylorr@openai.com","sentAt":"2026-10-26T23:49:40Z","receivedAt":"2026-10-26T23:49:40Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"The skipping negotiator sorts its queue by commit date, but callers\nwhich push commits onto that queue, such as `push_parent()`, can add a\nparent before parsing it. When an unparsed commit enters the queue, its\ndate is still zero, so old ref tips may sort ahead of it.\n\nFor example, suppose a topic branch is one commit ahead of master.\nSkipping offers the topic tip and skips master's tip. Master's parent\nthen waits behind old ref tips. With enough old refs, negotiation can\nend before we offer that recent common ancestor.\n\nParse commits in rev_list_push() before adding them to the queue, so\nthe queue uses their actual dates. If parsing fails, leave the commit\nout of the queue. Move the parsing from get_rev() to the insertion\npoint, where the date is needed to order the queue.\n\nSet SEEN only after parsing succeeds, following the default negotiator\nafter the preceding fix. push_parent() assumes that a commit marked\nSEEN but not POPPED has an entry in the queue. Marking an unparseable\nparent SEEN would violate that assumption and trigger a BUG() if another\nchild tried to push the same parent.\n\nUnlike the default negotiator, the skipping negotiator's version of\nrev_list_push() returns a pointer to an entry struct. Check for NULL in\npush_parent() before using that entry. The other two callers continue\nto ignore the return value.\n\nAdd tests for the ordering of recent ancestors relative to old ref tips\nand for two tips sharing a missing parent. In the latter case, fetching\nfrom a server that has the parent should retrieve it without triggering\nthe missing-entry BUG().\n\nSigned-off-by: Taylor Blau <ttaylorr@openai.com>\n---\n negotiator/skipping.c                |  9 +++-\n t/t5552-skipping-fetch-negotiator.sh | 63 ++++++++++++++++++++++++++++\n 2 files changed, 70 insertions(+), 2 deletions(-)\n\ndiff --git a/negotiator/skipping.c b/negotiator/skipping.c\nindex db90fa77b53..a7c7b8d9e94 100644\n--- a/negotiator/skipping.c\n+++ b/negotiator/skipping.c\n@@ -64,6 +64,10 @@ static int compare(const void *a_, const void *b_, void *data UNUSED)\n static struct entry *rev_list_push(struct data *data, struct commit *commit, int mark)\n {\n \tstruct entry *entry;\n+\n+\tif (repo_parse_commit(the_repository, commit))\n+\t\treturn NULL;\n+\n \tcommit->object.flags |= mark | SEEN;\n \n \tCALLOC_ARRAY(entry, 1);\n@@ -125,7 +129,7 @@ static void mark_common(struct data *data, struct commit *seen_commit)\n  * entry has the correct flags and ttl.\n  *\n  * This function returns 1 if an entry was found or created, and 0 otherwise\n- * (because the entry for this commit had already been popped).\n+ * (because the commit could not be parsed or its entry had already been popped).\n  */\n static int push_parent(struct data *data, struct entry *entry,\n \t\t       struct commit *to_push)\n@@ -152,6 +156,8 @@ static int push_parent(struct data *data, struct entry *entry,\n \t\t;\n \t} else {\n \t\tparent_entry = rev_list_push(data, to_push, 0);\n+\t\tif (!parent_entry)\n+\t\t\treturn 0;\n \t}\n \n \tif (entry->commit->object.flags & (COMMON | ADVERTISED)) {\n@@ -194,7 +200,6 @@ static const struct object_id *get_rev(struct data *data)\n \t\tif (!(commit->object.flags & COMMON) && !entry->ttl)\n \t\t\tto_send = commit;\n \n-\t\trepo_parse_commit(the_repository, commit);\n \t\tfor (p = commit->parents; p; p = p->next)\n \t\t\tparent_pushed |= push_parent(data, entry, p->item);\n \ndiff --git a/t/t5552-skipping-fetch-negotiator.sh b/t/t5552-skipping-fetch-negotiator.sh\nindex 96e224168eb..0efd2f2ac49 100755\n--- a/t/t5552-skipping-fetch-negotiator.sh\n+++ b/t/t5552-skipping-fetch-negotiator.sh\n@@ -223,4 +223,67 @@ test_expect_success 'do not send \"have\" with ancestors of commits that server AC\n \thave_sent b2.c1 b2.c0\n '\n \n+test_expect_success 'recent ancestors are sent before old ref tips' '\n+\trm -rf server client trace &&\n+\tgit init server &&\n+\ttest_tick=2000000000 &&\n+\ttest_commit -C server --no-tag base &&\n+\ttest_commit -C server --no-tag parent &&\n+\ttest_commit -C server --no-tag main &&\n+\tgit clone server client &&\n+\ttest_commit -C server --no-tag to_fetch &&\n+\n+\tgit -C client checkout -b topic &&\n+\ttest_commit -C client --no-tag topic &&\n+\n+\t# These old tips have known dates, but the recent parent has no ref.\n+\t# Without parsing parents before queueing them, its date is zero and\n+\t# all of these tips are sent first, delaying it past the first batch.\n+\tfor i in $(test_seq 32)\n+\tdo\n+\t\told=$(echo \"old $i\" |\n+\t\t\tGIT_AUTHOR_DATE=\"1000000000 +0000\" \\\n+\t\t\tGIT_COMMITTER_DATE=\"1000000000 +0000\" \\\n+\t\t\tgit -C client commit-tree HEAD^{tree}) &&\n+\t\tgit -C client update-ref refs/heads/old-$i \"$old\" || return 1\n+\tdone &&\n+\n+\ttest_config -C client fetch.negotiationalgorithm skipping &&\n+\ttrace_fetch client origin &&\n+\tgit -C client rev-parse HEAD HEAD~2 >expect &&\n+\tsed -n \"s/.*fetch> have //p\" trace | head -n 2 >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'missing parent shared by two tips is not treated as queued' '\n+\trm -rf server client trace &&\n+\tgit init server &&\n+\ttest_commit -C server base &&\n+\tparent=$(git -C server rev-parse HEAD) &&\n+\ttest_commit -C server to_fetch &&\n+\n+\tgit init client &&\n+\ttree=$(git -C client mktree </dev/null) &&\n+\t# Write the tips directly, bypassing the parent lookup in commit-tree.\n+\t# Both tips name a parent that only the server has.\n+\tfor i in 1 2\n+\tdo\n+\t\tcat >commit <<-EOF_COMMIT &&\n+\t\ttree $tree\n+\t\tparent $parent\n+\t\tauthor A U Thor <author@example.com> 1000000000 +0000\n+\t\tcommitter C O Mitter <committer@example.com> 1000000000 +0000\n+\n+\t\ttip $i\n+\t\tEOF_COMMIT\n+\t\ttip=$(git -C client hash-object -w -t commit --stdin <commit) &&\n+\t\tgit -C client update-ref refs/heads/tip-$i \"$tip\" || return 1\n+\tdone &&\n+\n+\ttest_config -C client fetch.negotiationalgorithm skipping &&\n+\ttrace_fetch client \"$(pwd)/server\" \\\n+\t\t--negotiation-tip=tip-1 --negotiation-tip=tip-2 &&\n+\tgit -C client cat-file -e \"$parent\"\n+'\n+\n test_done\n-- \n2.56.0.187.g9e9ae442333\n\n"}]}