git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 2/3] builtin/log: prefetch necessary blobs for `git cherry`

From
Derrick Stolee <stolee@gmail.com>
Date
Apr 27, 2026, 13:16 UTC
Message-ID
<a2fbb23d-0809-4a9d-8bf9-8ac0dc8ee054@gmail.com>
In-Reply-To
<a705852723fbe88e94ad3de1daba548dbce32211.1776472347.git.gitgitgadget@gmail.com>
On 4/17/2026 8:32 PM, Elijah Newren via GitGitGadget wrote:
> From: Elijah Newren <newren@gmail.com>

(I'm sorry that I'm reviewing out of order. This reply includes my feelings about patch 3 after reading both.)

Show 12 quoted lines
> +/*
> + * Enumerate blob OIDs from a single commit's diff, inserting them into blobs.
> + * Skips files whose userdiff driver explicitly declares binary status
> + * (drv->binary > 0), since patch-ID uses oid_to_hex() for those and
> + * never reads blob content.  Use userdiff_find_by_path() since
> + * diff_filespec_load_driver() is static in diff.c.
> + *
> + * Clean up with diff_queue_clear() (from diffcore.h).
> + */
> +static void collect_diff_blob_oids(struct commit *commit,
> +				   struct diff_options *opts,
> +				   struct oidset *blobs)

I think that this is generally a good idea, though I worry that having this hidden in builtin/log.c may not be the right long- term home.

I expect that we'll find more and more examples where we want to prefetch blobs in different operations, those that exist now and those that may be created in the future. It would be preferred if they could automatically take advantage of the logic already in diff_queued_diff_prefetch() within diffcore_std() in diff.c.

Ultimately, _this_ patch cares about a diff. Could we compute a "diff prep" computation using the core diff library instead of inventing a second queue of results for diffing?

Patch 3 cares about a "scan prep" which cares about loading all blobs for a given tree with respect to a pathspec. This is very similar to what a checkout would do, though it ultimately uses a form of diff to find out what change should be applied to the working directory. Perhaps 'git archive' is a better matching example.

I don't mean to make your series more complicated. I value what you're doing and can see how your current attention can be used to make further improvements later. By implementing things in a common location, then we can have later integrations add to the confidence in the feature through tests covering each user-facing use.

I'm not sure if it makes sense to attempt to create a universal library method that would be used by builtin/log.c _and_ diff.c, at least not right now. I'm most interested in having this logic be more reusable in the future without needing to move code across files.

Thanks, -Stolee

Previous: Elijah NewrenNext: Junio C Hamano
Message 14 of 29 in “Batch prefetching”
  1. 0/3 Batch prefetchingElijah Newren via GitGitGadget, Apr 16, 2026
  2. 1/3 patch-ids.h: add missing trailing parenthesis in documentation commentElijah Newren via GitGitGadget, Apr 16, 2026
  3. 2/3 builtin/log: prefetch necessary blobs for `git cherry`Elijah Newren via GitGitGadget, Apr 16, 2026
  4. Junio C HamanoApr 17, 2026
  5. Elijah NewrenApr 17, 2026
  6. 3/3 grep: prefetch necessary blobsElijah Newren via GitGitGadget, Apr 16, 2026
  7. 0/3 Batch prefetchingElijah Newren via GitGitGadget, Apr 18, 2026
  8. 1/3 patch-ids.h: add missing trailing parenthesis in documentation commentElijah Newren via GitGitGadget, Apr 18, 2026
  9. 2/3 builtin/log: prefetch necessary blobs for `git cherry`Elijah Newren via GitGitGadget, Apr 18, 2026
  10. Phillip WoodApr 19, 2026
  11. Elijah NewrenApr 21, 2026
  12. Phillip WoodApr 23, 2026
  13. Elijah NewrenApr 23, 2026
  14. Derrick StoleeApr 27, 2026
  15. Junio C HamanoMay 11, 2026
  16. Elijah NewrenMay 11, 2026
  17. Elijah NewrenMay 13, 2026
  18. Derrick StoleeMay 18, 2026
  19. 3/3 grep: prefetch necessary blobsElijah Newren via GitGitGadget, Apr 18, 2026
  20. Derrick StoleeApr 27, 2026
  21. Elijah NewrenMay 13, 2026
  22. 0/4 Batch prefetchingElijah Newren via GitGitGadget, May 14, 2026
  23. 1/4 promisor-remote: document caller filtering contractElijah Newren via GitGitGadget, May 14, 2026
  24. 2/4 patch-ids.h: add missing trailing parenthesis in documentation commentElijah Newren via GitGitGadget, May 14, 2026
  25. 3/4 builtin/log: prefetch necessary blobs for `git cherry`Elijah Newren via GitGitGadget, May 14, 2026
  26. 4/4 grep: prefetch necessary blobsElijah Newren via GitGitGadget, May 14, 2026
  27. Derrick StoleeMay 18, 2026
  28. Junio C HamanoMay 18, 2026
  29. Elijah NewrenMay 18, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.