{"thread":{"id":"66493","subject":"[PATCH] fetch: commit references fetched before backfilling tags","startedAt":"2026-10-08T22:10:37Z","lastAt":"2026-10-08T22:10:37Z","messageCount":1,"participants":["Karthik Nayak"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"554545","messageId":"20261009-799-shallow-fetch-with-tags-v1-1-379d61504af5@gmail.com","threadId":"66493","inReplyTo":null,"subject":"[PATCH] fetch: commit references fetched before backfilling tags","fromName":"Karthik Nayak","fromEmail":"karthik.188@gmail.com","sentAt":"2026-10-08T22:10:37Z","receivedAt":"2026-10-08T22:10:37Z","isPatch":true,"sender":{"key":"karthik.188@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1786334?v=4"},"body":"In 0e358de64a (fetch: use batched reference updates, 2025-05-19), the\nfetch code was modified to use batched updates to provide a good\nperformance improvement. Wherein batched updates were used to fetch both\nreferences and backfill tags.\n\nWhen using batched updates, the references aren't yet committed to disk\nwhen we start backfilling tags. This means in situations such as shallow\nfetching the negotiation during backfilling tags, the client doesn't\nhave any references to report in the 'have' section. Since backfilling\ndoesn't use a depth limit, this can cause the server to send all the\nobjects present in the repository.\n\nFix this by committing the previous batched update and initiating a new\none for backfilling tags. Also add a test which captures this regression.\n\nWhile this does make it a little slower than master, due to creation of\ntwo transactions, It is still faster than not using batched updates:\n\nBenchmark 1: fetch: many refs (refformat = reftable, refcount = 10000, revision = 0e358de64a9e014575d11ef884bfc9beb931e37f~1)\n  Time (mean ± σ):      1.468 s ±  0.041 s    [User: 0.839 s, System: 0.587 s]\n  Range (min … max):    1.427 s …  1.558 s    10 runs\n\nBenchmark 2: fetch: many refs (refformat = reftable, refcount = 10000, revision = HEAD)\n  Time (mean ± σ):      84.4 ms ±   1.7 ms    [User: 60.9 ms, System: 25.8 ms]\n  Range (min … max):    81.4 ms …  88.6 ms    29 runs\n\nSummary\n  fetch: many refs (refformat = reftable, refcount = 10000, revision = HEAD) ran\n   17.38 ± 0.61 times faster than fetch: many refs (refformat = reftable, refcount = 10000, revision = 0e358de64a9e014575d11ef884bfc9beb931e37f~1)\n\nReported-by: Mitja Bezenšek <mitja.bezensek@login5.org>\nSigned-off-by: Karthik Nayak <karthik.188@gmail.com>\n---\nThe issue was reported by Mitja Bezenšek on GitLab's Git repo [1].\n\n[1]: https://gitlab.com/gitlab-org/git/-/work_items/799\n---\n builtin/fetch.c  | 21 +++++++++++++++++++++\n t/t5510-fetch.sh | 17 +++++++++++++++++\n 2 files changed, 38 insertions(+)\n\ndiff --git a/builtin/fetch.c b/builtin/fetch.c\nindex b2decc6cfd..68b04d0f8a 100644\n--- a/builtin/fetch.c\n+++ b/builtin/fetch.c\n@@ -2076,6 +2076,27 @@ static int do_fetch(struct transport *transport,\n \t\tstruct ref *tags_ref_map = NULL, **tail = &tags_ref_map;\n \n \t\tfind_non_local_tags(remote_refs, transaction, &tags_ref_map, &tail);\n+\n+\t\t/*\n+\t\t * Backfilling tags has no depth limit. If we don't commit\n+\t\t * the fetched references, the backfill will report no refs\n+\t\t * in the 'have' section of the negotiation. This can cause\n+\t\t * the server to send all objects.\n+\t\t */\n+\t\tif (tags_ref_map && !atomic_fetch) {\n+\t\t\tretcode |= commit_ref_transaction(&transaction, false,\n+\t\t\t\t\t\t\t  transport->remote->name,\n+\t\t\t\t\t\t\t  &rejected_refs, &err);\n+\n+\t\t\ttransaction = ref_store_transaction_begin(get_main_ref_store(the_repository),\n+\t\t\t\t\t\t\t\t  REF_TRANSACTION_ALLOW_FAILURE, &err);\n+\t\t\tif (!transaction) {\n+\t\t\t\tfree_refs(tags_ref_map);\n+\t\t\t\tretcode = -1;\n+\t\t\t\tgoto cleanup;\n+\t\t\t}\n+\t\t}\n+\n \t\tif (tags_ref_map) {\n \t\t\t/*\n \t\t\t * If backfilling of tags fails then we want to tell\ndiff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh\nindex 300bd5396d..81865c1ecc 100755\n--- a/t/t5510-fetch.sh\n+++ b/t/t5510-fetch.sh\n@@ -1942,6 +1942,23 @@ test_expect_success \"backfill tags when providing a refspec\" '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'shallow fetch does not fetch objects again for tags' '\n+\ttest_when_finished rm -rf source target trace &&\n+\n+\tgit init source &&\n+\ttest_commit_bulk -C source 10 &&\n+\tgit -C source tag -a tag -m tag HEAD~2 &&\n+\tHEAD_OID=$(git -C source rev-parse HEAD) &&\n+\n+\tgit init target &&\n+\tgit -C target remote add origin ../source &&\n+\tGIT_TEST_PROTOCOL_VERSION=2 GIT_TRACE_PACKET=$(pwd)/trace \\\n+\t\tgit -C target fetch --depth 5 origin &&\n+\n+\ttest $(grep -c \"fetch> command=fetch\" trace) -gt 2 &&\n+\ttest $(grep -c \"fetch> have $HEAD_OID\" trace) -eq 2\n+'\n+\n test_expect_success REFFILES \"FETCH_HEAD is updated even if ref updates fail\" '\n \ttest_when_finished rm -rf base repo &&\n \n\n---\nbase-commit: 6de20f6092dcf9bdb1c8efe03db4b70c82b423dd\nchange-id: 20261007-799-shallow-fetch-with-tags-d62b3efb29f0\n\n\nThanks\n- Karthik\n\n\n"}]}