git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 5/5] path-walk: support wildcard pathspecs for blob filtering

From
Derrick Stolee <stolee@gmail.com>
Date
Mar 23, 2026, 01:33 UTC
Message-ID
<bf0cc84b-b4ea-4100-87b6-d29777155037@gmail.com>
In-Reply-To
<443e5b25-46a1-4990-96b3-122e66d81828@gmail.com>
On 3/18/26 9:16 AM, Derrick Stolee wrote:
Show 33 quoted lines
> On 3/17/2026 6:19 PM, Junio C Hamano wrote:
>> "Derrick Stolee via GitGitGadget" <gitgitgadget@gmail.com> writes:
>>
>>> From: Derrick Stolee <stolee@gmail.com>
>>>
>>> Previously, walk_objects_by_path() silently ignored pathspecs containing
>>> wildcards or magic by clearing them. This caused all blobs to be
>>> downloaded regardless of the given pathspec. Wildcard pathspecs like
>>> "d/file.*.txt" are useful for narrowing which blobs to process (e.g.,
>>> during 'git backfill').
>>>
>>> Support wildcard pathspecs by making three changes:
>>>
>>>   1. Add an 'exact_pathspecs' flag to path_walk_context. When the
>>>      pathspec has no wildcards or magic, set this flag and use the
>>>      existing fast-path prefix matching in add_tree_entries(). When
>>>      wildcards are present, skip that block since prefix matching
>>>      cannot handle glob patterns.
>>>
>>>   2. Disable revision-level commit pruning (revs->prune = 0) for
>>>      wildcard pathspecs. The revision walk uses the pathspec to filter
>>>      commits via TREESAME detection. For exact prefix pathspecs this
>>>      works well, but wildcard pathspecs may fail to match through
>>>      TREESAME because fnmatch with WM_PATHNAME does not cross directory
>>>      boundaries. Disabling pruning ensures all commits are visited and
>>>      their trees are available for the path-walk to filter.
>>
>> Hmph, I wonder how significant an impact does it have on the
>> performance that we have to disable pruning here.  With the bog
>> standard tree traversal, wouldn't tree_entry_interesting() already
>> be capable of doing this, even with fnmatch / WM_PATHNAME ?
> 
> I will explore what's possible here and see what I can do.

I must have needed the 'revs->prune = 0' at some point during development and left it even though it isn't actually necessary. Leaving it implicitly at '1' should indeed be faster due to traversing fewer commits and parsing fewer trees while still reaching all necessary blobs.

Only changes 1 and 3 are necessary.

Thanks, -Stolee

Previous: Derrick StoleeNext: Junio C Hamano
Message 21 of 46 in “backfill: accept revision arguments”
  1. 0/5 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 17, 2026
  2. 1/5 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 17, 2026
  3. Junio C HamanoMar 17, 2026
  4. 2/5 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 17, 2026
  5. 3/5 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 17, 2026
  6. Junio C HamanoMar 17, 2026
  7. Kristoffer HaugsbakkMar 18, 2026
  8. Derrick StoleeMar 23, 2026
  9. Patrick SteinhardtMar 19, 2026
  10. Derrick StoleeMar 23, 2026
  11. 4/5 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 17, 2026
  12. Junio C HamanoMar 17, 2026
  13. Derrick StoleeMar 18, 2026
  14. Patrick SteinhardtMar 19, 2026
  15. Patrick SteinhardtMar 19, 2026
  16. Patrick SteinhardtMar 19, 2026
  17. Derrick StoleeMar 23, 2026
  18. 5/5 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 17, 2026
  19. Junio C HamanoMar 17, 2026
  20. Derrick StoleeMar 18, 2026
  21. Derrick StoleeMar 23, 2026
  22. Junio C HamanoMar 17, 2026
  23. Patrick SteinhardtMar 19, 2026
  24. Derrick StoleeMar 19, 2026
  25. Patrick SteinhardtMar 20, 2026
  26. 0/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 23, 2026
  27. 1/6 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 23, 2026
  28. 2/6 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 23, 2026
  29. 3/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 23, 2026
  30. Patrick SteinhardtMar 24, 2026
  31. Derrick StoleeMar 26, 2026
  32. 4/6 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 23, 2026
  33. Patrick SteinhardtMar 24, 2026
  34. Derrick StoleeMar 26, 2026
  35. 5/6 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 23, 2026
  36. 6/6 t5620: test backfill's unknown argument handlingDerrick Stolee via GitGitGadget, Mar 23, 2026
  37. Junio C HamanoMar 23, 2026
  38. Derrick StoleeMar 23, 2026
  39. 0/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 26, 2026
  40. 1/6 revision: include object-name.hDerrick Stolee via GitGitGadget, Mar 26, 2026
  41. 2/6 t5620: prepare branched repo for revision testsDerrick Stolee via GitGitGadget, Mar 26, 2026
  42. 3/6 backfill: accept revision argumentsDerrick Stolee via GitGitGadget, Mar 26, 2026
  43. 4/6 backfill: work with prefix pathspecsDerrick Stolee via GitGitGadget, Mar 26, 2026
  44. 5/6 path-walk: support wildcard pathspecs for blob filteringDerrick Stolee via GitGitGadget, Mar 26, 2026
  45. 6/6 t5620: test backfill's unknown argument handlingDerrick Stolee via GitGitGadget, Mar 26, 2026
  46. Patrick SteinhardtMar 27, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.