git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH] packfile: fix perf regression with many packs

From
Ben Knoble <ben.knoble@gmail.com>
Date
Aug 12, 2026, 22:29 UTC
Message-ID
<6EA76E66-E80C-4F19-8806-FAE8294ACFB7@gmail.com>
In-Reply-To
<pull.2202.git.1786561870638.gitgitgadget@gmail.com>
Show 37 quoted lines
> Le 12 août 2026 à 15:15, Johannes Schindelin via GitGitGadget <gitgitgadget@gmail.com> a écrit :
> 
> From: Johannes Schindelin <johannes.schindelin@gmx.de>
> 
> Since 589127caa730 (packfile: move list of packs into the packfile
> store, 2025-10-30), there is a performance regression when many
> packfiles need to be loaded: `packfile_store_add_pack()` now calls
> `packfile_list_remove_internal()` to detect whether the packfile was
> _already_ in the list, if if so, move it to the end of the list. This
> function linearly scans the existing list before every insertion. Newly
> loading N packs therefore has complexity O(N²).
> 
> In one reported use case (https://github.com/microsoft/git/issues/970),
> N equals 37,815 and caused a slow-down of a simple `git rev-parse
> --short HEAD` (which is regularly executed as part of `GIT_PS1`) from
> 0.4s to 4.5s. In another, heavily exercised CI scenario, clone times
> increased from under 2 minutes to over half an hour.
> 
> Let's fix this by establishing a fast path for known-new packfiles.
> 
> The keen reader will note that there is currently only a single,
> "known-new" caller of the `packfile_list_append()` function, and wonder
> why not simply remove this check whether the packfile already exists in
> the list? Originally, when above-mentioned commit introduced that logic,
> there was a second caller in `prepare_midx()`, which would have required
> that check, but that caller was removed in 6aff1f25a046 (packfile:
> always add packfiles to MRU when adding a pack, 2025-10-30). Still, the
> function is declared in a header file, and to avoid any problems with
> in-flight or downstream callers, it is safer to extend the signature to
> be explicit whether or not to skip that check.
> 
> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
> ---
>    packfile: fix perf regression with many packs
> 
>    This issue was spotted by a Microsoft Git user with the massive amount
>    of packfiles typical of an average, long-running monorepo checkout.
As a different kind of intermediate solution, would turning on maintenance for that user’s checkout help? (Not sure that would help CI clone times unless the server repacks, of course.)
Previous: Johannes SchindelinNext: Johannes Schindelin
Message 8 of 24 in “packfile: fix perf regression with many packs”
  1. packfile: fix perf regression with many packsJohannes Schindelin via GitGitGadget, Aug 12, 2026
  2. Junio C HamanoAug 12, 2026
  3. Jeff KingAug 12, 2026
  4. Patrick SteinhardtAug 13, 2026
  5. Johannes SchindelinAug 13, 2026
  6. Jeff KingAug 13, 2026
  7. Johannes SchindelinAug 13, 2026
  8. Ben KnobleAug 12, 2026
  9. Johannes SchindelinAug 13, 2026
  10. Ben KnobleAug 13, 2026
  11. Patrick SteinhardtAug 13, 2026
  12. Johannes SchindelinAug 13, 2026
  13. Patrick SteinhardtAug 13, 2026
  14. Johannes SchindelinAug 13, 2026
  15. Patrick SteinhardtAug 13, 2026
  16. Junio C HamanoAug 13, 2026
  17. Jeff KingAug 13, 2026
  18. Patrick SteinhardtAug 14, 2026
  19. Jeff KingAug 14, 2026
  20. Patrick SteinhardtAug 17, 2026
  21. Jeff KingAug 17, 2026
  22. packfile: fix perf regression with many packsJohannes Schindelin via GitGitGadget, Aug 13, 2026
  23. Patrick SteinhardtAug 14, 2026
  24. Junio C HamanoAug 14, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.