git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 0/3] Batch prefetching

From
Elijah Newren via GitGitGadget <gitgitgadget@gmail.com>
Date
Apr 16, 2026, 22:48 UTC
Message-ID
<pull.2089.git.1776379694.gitgitgadget@gmail.com>

Partial clones provide a trade-off for users: avoid downloading blobs upfront, at the expense of needing to download them later as they run other commands. This tradeoff can sometimes incur a more severe cost than expected, particularly if needed blobs are discovered as they are accessed, resulting in downloading blobs one at a time. Some commands like checkout, diff, and merge do batch prefetches of necessary blobs, since that can dramatically reduce the pain of on-demand loading. Extend this ability to two more commands: cherry and grep.

This series was spurred by a report where git cherry jobs were each doing hundreds of single-blob fetches, at a cost of 3s each. Batching those downloads should dramatically speed up their jobs. (And I decided to fix up git grep similarly while at it.)

I'll also note that git backfill with revisions and/or pathspecs could also improve things for these users, but since backfill is a manual command users would have to run and requires users to try to figure out which data is needed (a challenge in the case of cherry), it still makes sense to provide smarter behavior for folks who don't choose to manually run backfill.

Also, correct a documentation typo I noticed in patch-ids.h (related to code I was using for the git cherry fixes) as a preparatory fixup.

Elijah Newren (3):
  patch-ids.h: add missing trailing parenthesis in documentation comment
  builtin/log: prefetch necessary blobs for `git cherry`
  grep: prefetch necessary blobs
 builtin/grep.c                                | 142 ++++++++++++
 builtin/log.c                                 | 125 +++++++++++
 investigations/cherry-prefetch-design-spec.md | 210 ++++++++++++++++++
 patch-ids.h                                   |   2 +-
 t/t3500-cherry.sh                             |  18 ++
 t/t7810-grep.sh                               |  35 +++
 6 files changed, 531 insertions(+), 1 deletion(-)
 create mode 100644 investigations/cherry-prefetch-design-spec.md
base-commit: 9f223ef1c026d91c7ac68cc0211bde255dda6199
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2089%2Fnewren%2Fbatch-prefetching-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2089/newren/batch-prefetching-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2089
-- 
gitgitgadget
Next: Elijah Newren via GitGitGadget
Message 1 of 29 in “Batch prefetching”
  1. 0/3 Batch prefetchingElijah Newren via GitGitGadget, Apr 16, 2026
  2. 1/3 patch-ids.h: add missing trailing parenthesis in documentation commentElijah Newren via GitGitGadget, Apr 16, 2026
  3. 2/3 builtin/log: prefetch necessary blobs for `git cherry`Elijah Newren via GitGitGadget, Apr 16, 2026
  4. Junio C HamanoApr 17, 2026
  5. Elijah NewrenApr 17, 2026
  6. 3/3 grep: prefetch necessary blobsElijah Newren via GitGitGadget, Apr 16, 2026
  7. 0/3 Batch prefetchingElijah Newren via GitGitGadget, Apr 18, 2026
  8. 1/3 patch-ids.h: add missing trailing parenthesis in documentation commentElijah Newren via GitGitGadget, Apr 18, 2026
  9. 2/3 builtin/log: prefetch necessary blobs for `git cherry`Elijah Newren via GitGitGadget, Apr 18, 2026
  10. Phillip WoodApr 19, 2026
  11. Elijah NewrenApr 21, 2026
  12. Phillip WoodApr 23, 2026
  13. Elijah NewrenApr 23, 2026
  14. Derrick StoleeApr 27, 2026
  15. Junio C HamanoMay 11, 2026
  16. Elijah NewrenMay 11, 2026
  17. Elijah NewrenMay 13, 2026
  18. Derrick StoleeMay 18, 2026
  19. 3/3 grep: prefetch necessary blobsElijah Newren via GitGitGadget, Apr 18, 2026
  20. Derrick StoleeApr 27, 2026
  21. Elijah NewrenMay 13, 2026
  22. 0/4 Batch prefetchingElijah Newren via GitGitGadget, May 14, 2026
  23. 1/4 promisor-remote: document caller filtering contractElijah Newren via GitGitGadget, May 14, 2026
  24. 2/4 patch-ids.h: add missing trailing parenthesis in documentation commentElijah Newren via GitGitGadget, May 14, 2026
  25. 3/4 builtin/log: prefetch necessary blobs for `git cherry`Elijah Newren via GitGitGadget, May 14, 2026
  26. 4/4 grep: prefetch necessary blobsElijah Newren via GitGitGadget, May 14, 2026
  27. Derrick StoleeMay 18, 2026
  28. Junio C HamanoMay 18, 2026
  29. Elijah NewrenMay 18, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.