git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC PATCH 1/2] fetch-prune: optimize dangling-ref reporting

From
Junio C Hamano <gitster@pobox.com>
Date
Jun 18, 2025, 21:50 UTC
Message-ID
<xmqqzfe4d8hy.fsf@gitster.g>
In-Reply-To
<20250618211024.2332525-2-phil.hord@gmail.com>
Phil Hord <phil.hord@gmail.com> writes:
Show 19 quoted lines
> From: Phil Hord <phil.hord@gmail.com>
>
> When pruning during `git fetch` we check each pruned ref against the
> ref_store one at a time to decide whether to report it as dangling.
> This causes every local ref to be scanned for each ref being pruned.
>
> If there are N refs in the repo and M refs being pruned, this code is
> O(M*N). However, `git remote prune` uses a very similar function that
> is only O(N*log(M)).
>
> Remove the wasteful ref scanning for each pruned ref and use the faster
> version already available in refs_warn_dangling_symrefs.
>
> In a repo with 126,000 refs, where I was pruning 28,000 refs, this
> code made about 3.6 billion calls to strcmp and consumed 410 seconds
> of CPU. (Invariably in that time, my remote would timeout and the
> fetch would fail anyway.)
>
> After this change, the same operation completes in under 4 seconds.
Nice.
Previous: Phil HordNext: Jacob Keller
Message 3 of 14 in “fetch --prune performance problem”
  1. 0/2 fetch --prune performance problemPhil Hord, Jun 18, 2025
  2. 1/2 fetch-prune: optimize dangling-ref reportingPhil Hord, Jun 18, 2025
  3. Junio C HamanoJun 18, 2025
  4. Jacob KellerJun 18, 2025
  5. Jeff KingJun 19, 2025
  6. Lidong YanJun 19, 2025
  7. Lidong YanJun 19, 2025
  8. 2/2 refs: remove old refs_warn_dangling_symrefPhil Hord, Jun 18, 2025
  9. Jacob KellerJun 18, 2025
  10. Jeff KingJun 19, 2025
  11. Junio C HamanoJun 19, 2025
  12. Jacob KellerJun 23, 2025
  13. Junio C HamanoJun 23, 2025
  14. Jacob KellerJun 23, 2025

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.