git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v5 1/3] builtin/grep.c: add --sparse option

From
Victoria Dye <vdye@github.com>
Date
Sep 18, 2022, 19:52 UTC
Message-ID
<cafcedba-96a2-cb85-d593-ef47c8c8397c@github.com>
In-Reply-To
<CABPp-BEOVGfgmAMGCjP6Q3k-t=C1tL=f27buhiCiL-Wv0eDF_A@mail.gmail.com>
Elijah Newren wrote:
Show 55 quoted lines
> == Overall ==
> 
> For existing querying commands (just ls-files), `--sparse` already
> means restrict to the sparse cone.  If we keep using the existing flag
> names, grep should follow suit.
> 
> For existing modification commands already released (add, rm), the
> fact that the command is modifying actually gives a different way to
> interpret things such that it's not clear `--sparse` was even a
> problem.  However, perhaps the name of the flag is bad just because
> there are multiple ways to view it and those who view it one way will
> see it as counter-intuitive.
> 
> == Flag rename? ==
> 
> There's another reason to potentially rename the flag.  We already
> have `--sparse` and `--dense` flags for rev-list and friends.  So,
> when we want to enable those other commands to restrict to the
> sparsity patterns, we probably need a different name.  So, perhaps, we
> should rename our `--sparse/--dense` to `--restrict/--no-restrict`.
> Such a rename would also likely clear up the ambiguity about which way
> to interpret the command for the add & rm commands (though it'd pick
> the second one and suggest we were using the wrong name after all).
> 
> (There are also two other commands that use `--sparse` -- pack-objects
> and show-branch, though in a much different way and neither would ever
> be affected by our new --sparse/--dense/--restrict/--no-restrict
> flags.)
> 
> Other names are also possible.  Any suggestions?
> 
> == global flag vs subcommand flags ==
> 
> Do we want to make --[no-]restrict a flag for each subcommand, or just
> make it a global git flag?  I kind of think it'd make sense to do the
> latter
> 
> == Defaults ==
> 
> As discussed before, we probably want querying commands (ls-files,
> grep, log, etc.) to default to --no-restrict for now, since we are
> otherwise slowly changing the defaults.  We may want to swap that
> default in the future.
> 
> However, for modification commands, I think we want the default to be
> --restrict, regardless of the default for querying commands.  There
> are some potentially very negative surprises for users if we don't,
> and those surprises will be delayed rather than occur at the time the
> user runs the command.  In fact, those negative surprises are likely
> why those commands were the first to gain an option controlling
> whether they operated on paths outside the sparsity specification.
> (Also, the modification commands print a warning if they could have
> affected other files but didn't due the the default of restricting, so
> I think we have their default correct, even if the flag name is
> suboptimal.)

One of the things I've found myself a bit frustrated with while working on these sparse index integrations is that we haven't had a clear set of guidelines for times when we need to make UI/UX changes relating to 'sparse-checkout' compatibility. I think what you've outlined here is a good start to a larger discussion on the topic, but in the middle of this series might not be the best place for that discussion (at least in terms of preserving for later reference).

Elijah, would you be interested in compiling your thoughts into a document in 'Documentation/technical'? If not, Stolee or I could do it. If we could settle on some guidelines (option names, behavior, etc.) for better incorporating 'sparse-checkout' support into existing commands, it'd make future sparse index work substantially easier for everyone involved.

As for this series, I think the best way to move the sparse index work along is to drop this patch ("builtin/grep.c: add --sparse option") altogether. Shaoxuan's updates in patch 3 [1] make 'git grep' sparse index-compatible for *all* invocations (not just those without '--sparse'), so we don't need the new option for sparse index compatibility. It can then be re-introduced later (possibly modified) in a series dedicated to unifying the sparse-checkout UX.

[1] https://lore.kernel.org/git/20220908001854.206789-4-shaoxuan.yuan02@gmail.com/
Previous: Elijah NewrenNext: Junio C Hamano
Message 44 of 69 in “grep: integrate with sparse index”
  1. 0/2 grep: integrate with sparse indexShaoxuan Yuan, Aug 17, 2022
  2. 1/2 builtin/grep.c: add --sparse optionShaoxuan Yuan, Aug 17, 2022
  3. Derrick StoleeAug 17, 2022
  4. Junio C HamanoAug 17, 2022
  5. Victoria DyeAug 17, 2022
  6. Derrick StoleeAug 17, 2022
  7. Junio C HamanoAug 17, 2022
  8. Elijah NewrenAug 17, 2022
  9. Shaoxuan YuanAug 24, 2022
  10. Derrick StoleeAug 24, 2022
  11. 2/2 builtin/grep.c: integrate with sparse indexShaoxuan Yuan, Aug 17, 2022
  12. Derrick StoleeAug 17, 2022
  13. Shaoxuan YuanAug 24, 2022
  14. Derrick StoleeAug 25, 2022
  15. Derrick StoleeAug 17, 2022
  16. 0/2 grep: integrate with sparse indexShaoxuan Yuan, Aug 29, 2022
  17. 2/2 builtin/grep.c: integrate with sparse indexShaoxuan Yuan, Aug 29, 2022
  18. Derrick StoleeAug 30, 2022
  19. 1/2 builtin/grep.c: add --sparse optionShaoxuan Yuan, Aug 29, 2022
  20. 0/3 grep: integrate with sparse indexShaoxuan Yuan, Sep 1, 2022
  21. 1/3 builtin/grep.c: add --sparse optionShaoxuan Yuan, Sep 1, 2022
  22. 2/3 builtin/grep.c: integrate with sparse indexShaoxuan Yuan, Sep 1, 2022
  23. 3/3 builtin/grep.c: walking tree instead of expanding index with --sparseShaoxuan Yuan, Sep 1, 2022
  24. Derrick StoleeSep 1, 2022
  25. Shaoxuan YuanSep 1, 2022
  26. Junio C HamanoSep 1, 2022
  27. Junio C HamanoSep 1, 2022
  28. Shaoxuan YuanSep 1, 2022
  29. Shaoxuan YuanSep 1, 2022
  30. Victoria DyeSep 2, 2022
  31. Shaoxuan YuanSep 2, 2022
  32. 0/3 grep: integrate with sparse indexShaoxuan Yuan, Sep 3, 2022
  33. 1/3 builtin/grep.c: add --sparse optionShaoxuan Yuan, Sep 3, 2022
  34. 2/3 builtin/grep.c: integrate with sparse indexShaoxuan Yuan, Sep 3, 2022
  35. 3/3 builtin/grep.c: walking tree instead of expanding index with --sparseShaoxuan Yuan, Sep 3, 2022
  36. Junio C HamanoSep 3, 2022
  37. Shaoxuan YuanSep 8, 2022
  38. 0/3 grep: integrate with sparse indexShaoxuan Yuan, Sep 8, 2022
  39. 1/3 builtin/grep.c: add --sparse optionShaoxuan Yuan, Sep 8, 2022
  40. Victoria DyeSep 10, 2022
  41. Elijah NewrenSep 14, 2022
  42. Junio C HamanoSep 15, 2022
  43. Elijah NewrenSep 18, 2022
  44. Victoria DyeSep 18, 2022
  45. Junio C HamanoSep 19, 2022
  46. Shaoxuan YuanSep 19, 2022
  47. Ævar Arnfjörð BjarmasonSep 19, 2022
  48. Elijah NewrenSep 20, 2022
  49. Shaoxuan YuanSep 17, 2022
  50. Elijah NewrenSep 18, 2022
  51. Shaoxuan YuanSep 19, 2022
  52. Shaoxuan YuanSep 17, 2022
  53. 2/3 builtin/grep.c: integrate with sparse indexShaoxuan Yuan, Sep 8, 2022
  54. 3/3 builtin/grep.c: walking tree instead of expanding index with --sparseShaoxuan Yuan, Sep 8, 2022
  55. Junio C HamanoSep 8, 2022
  56. Derrick StoleeSep 8, 2022
  57. Junio C HamanoSep 8, 2022
  58. Shaoxuan YuanSep 8, 2022
  59. Derrick StoleeSep 9, 2022
  60. Junio C HamanoSep 13, 2022
  61. Victoria DyeSep 10, 2022
  62. 0/1 grep: integrate with sparse indexShaoxuan Yuan, Sep 23, 2022
  63. 1/1 builtin/grep.c: integrate with sparse indexShaoxuan Yuan, Sep 23, 2022
  64. Junio C HamanoSep 23, 2022
  65. Junio C HamanoSep 23, 2022
  66. Junio C HamanoSep 26, 2022
  67. Derrick StoleeSep 23, 2022
  68. Victoria DyeSep 23, 2022
  69. Junio C HamanoSep 23, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.