From: Derrick Stolee Date: Mon, 27 Apr 2026 12:59:48 GMT Subject: Re: [PATCH v2 3/3] grep: prefetch necessary blobs Message-ID: <31763514-2602-4d8e-ac25-70590f090947@gmail.com> In-Reply-To: <8fbfe69bc4d0c6166967986f24861ffa393ed7cf.1776472347.git.gitgitgadget@gmail.com> On 4/17/2026 8:32 PM, Elijah Newren via GitGitGadget wrote: > From: Elijah Newren > > In partial clones, `git grep` fetches necessary blobs on-demand one > at a time, which can be very slow. In partial clones, add an extra > preliminary walk over the tree similar to grep_tree() which collects > the blobs of interest, and then prefetches them. A log of the code is about walking trees to find blobs matching the input pathspec, with this being the core method: > +static void collect_blob_oids_for_tree(struct repository *repo, > + const struct pathspec *pathspec, > + struct tree_desc *tree, > + struct strbuf *base, > + int tn_len, > + struct oidset *blob_oids) And in your test, you set up a repo to have three blobs with matches in two of the files: > +test_expect_success 'grep of revision in partial clone does bulk prefetch' ' > + test_when_finished "rm -rf grep-partial-src grep-partial" && > + > + git init grep-partial-src && > + ( > + cd grep-partial-src && > + git config uploadpack.allowfilter 1 && > + git config uploadpack.allowanysha1inwant 1 && > + echo "needle in haystack" >searchme && > + echo "no match here" >other && > + mkdir subdir && > + echo "needle again" >subdir/deep && > + git add . && > + git commit -m "initial" > + ) && But then the command downloads all of the blobs, not using a pathspec: > + # grep HEAD should batch-prefetch all blobs in one request. > + GIT_TRACE2_EVENT="$(pwd)/grep-trace" \ > + git -C grep-partial grep -c "needle" HEAD >result && > + > + # Should find matches in two files. > + test_line_count = 2 result && > + > + # Should have prefetched all 3 objects at once > + test_trace2_data promisor fetch_count 3 +' I think your code is correct, but I'd like to see a test here that demonstrates a pathspec filter on the 'grep' command to help filter out a blob that has a matching string. Perhaps something like: * matches.txt (has needle) * nomatch.txt (does not have needle) * matches.md (has needle) and then 'git grep -c "needle" HEAD -- *.txt' would download two blobs and find one match. A second run without the pathspec would download one blob and find two matches. Does that make sense as a test? Thanks, -Stolee