git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 4/7] sparse-checkout: error or warn when given individual files

From
Elijah Newren <newren@gmail.com>
Date
Feb 15, 2022, 04:17 UTC
Message-ID
<CABPp-BE8aG3R4ASngqwyvRemp5WW=O0UWWSTesJ2hoch_av_kQ@mail.gmail.com>
In-Reply-To
<72d39c4a-fd16-846c-2a5e-8b9ba0c1ab07@github.com>
On Mon, Feb 14, 2022 at 7:56 AM Derrick Stolee <derrickstolee@github.com> wrote:
Show 21 quoted lines
>
> On 2/12/2022 7:39 PM, Elijah Newren via GitGitGadget wrote:
> > From: Elijah Newren <newren@gmail.com>
> >
> > The set and add subcommands accept multiple positional arguments.
> > The meaning of these arguments differs slightly in the two modes:
> >
> > Cone mode only accepts directories.  If given a file, it would
> > previously treat it as a directory, causing not just the file itself to
> > be included but all sibling files as well -- likely against users'
> > expectations.  Throw an error if the specified path is a file in the
> > index.  Provide a --skip-checks argument to allow users to override
> > (e.g. for the case when the given path IS a directory on another
> > branch).
>
> I agree that this is likely to be an improvement for users. The
> sparse-checkout builtin isn't integrated with the sparse index
> yet. At least not integrated upstream: we have commits in microsoft/git
> that we plan to send when other things in flight are merged. This
> change likely introduces a new opportunity for the index to expand,
> so I will keep that in mind when upstreaming.

Actually, I thought about that during development, and my presumption was that we would not expand the index. We've survived a few years without reporting any argument errors to the user and folks seem to usually get things right, so while I think it adds value to report on likely errors, I don't think it's important for us to catch and warn on every potential misuse. I think the probable errors are the ones where they specify a <file> that exists in both the working tree and index. The remaining ones are less probable, and also possibly quite expensive to catch. I'm not sure it's worth the cost to try to report those.

Show 21 quoted lines
> > Non-cone mode accepts general gitignore patterns.  However, it has an
> > O(N*M) performance baked into its design, where all N index files must
> > be matched against all M sparse-checkout patterns with EVERY call to
> > unpack_trees() that updates the working tree.  As such, it is important
> > to keep the number of patterns small, and thus we should warn users to
> > prefer passing directories and more generic glob patterns to get the
> > paths they want instead of listing each individual file.  (The
> > --skip-checks argument can also be used to bypass this warning.)  Also,
> > even when users do want to specify individual files, they will often
> > want to do so by providing a leading '/' (to avoid selecting the same
> > filename in all subdirectories), in which case this error message would
> > never trigger anyway.
>
> I think the case of "I want only one file from this directory" and "I
> want files with the given name pattern" are the main reason to still
> use non-cone-mode. Users with this need usually have a directory full
> of large files, and they choose which of those large files they need
> using sparse-checkout. The repository reorganization required to use
> cone mode for this use is perhaps too great (or they haven't thought
> about doing it). For this reason, I would prefer that we do not do
> these checks when not in cone mode.

If they "only want one file from this directory", isn't the correct way to specify that by mentioning the path with a leading slash? Otherwise, they'd potentially grab files with similar names from many directories, right? So, even in that usecase, we should still error out if they specify a <filename> rather than /<filename>. Perhaps my reasoning should lead with that and I should fix up the warning message a bit, but I still think we should probably give a warning even for those who are explicitly wanting the usecase you mention.

Also, note this is a warning and not an error -- and the warning can be suppressed with --skip-checks.

Show 19 quoted lines
> > +test_expect_success 'by default, cone mode will error out when passed files' '
> > +     git -C repo sparse-checkout reapply --cone &&
> > +     test_must_fail git -C repo sparse-checkout add .gitignore 2>error &&
> > +
> > +     grep ".gitignore.*is not a directory" error
> > +'
> > +
> > +test_expect_success 'by default, non-cone mode will warn on individual files' '
> > +     git -C repo sparse-checkout reapply --no-cone &&
> > +     git -C repo sparse-checkout add .gitignore 2>warning &&
> > +
> > +     grep "passing directories or less specific patterns is recommended" warning
> > +'
>
> So I would expect this second test to have
>
>         test_must_be_empty warning
>
> to show that no warning occurs when specifying a file in non-cone-mode.
or perhaps
grep "please specify a leading slash to select a single file" warning
?
Previous: Derrick StoleeNext: Derrick Stolee
Message 12 of 70 in “RFC: sparse checkout: make --cone mode the default, and check add/set argument validity”
  1. 0/7 RFC: sparse checkout: make --cone mode the default, and check add/set argument validityElijah Newren via GitGitGadget, Feb 13, 2022
  2. 1/7 sparse-checkout: correct reapply's handling of optionsElijah Newren via GitGitGadget, Feb 13, 2022
  3. 3/7 sparse-checkout: pay attention to prefix for {set, add}Elijah Newren via GitGitGadget, Feb 13, 2022
  4. Derrick StoleeFeb 14, 2022
  5. Elijah NewrenFeb 15, 2022
  6. Derrick StoleeFeb 15, 2022
  7. 2/7 sparse-checkout: correctly set non-cone mode when expectedElijah Newren via GitGitGadget, Feb 13, 2022
  8. Derrick StoleeFeb 14, 2022
  9. Elijah NewrenFeb 15, 2022
  10. 4/7 sparse-checkout: error or warn when given individual filesElijah Newren via GitGitGadget, Feb 13, 2022
  11. Derrick StoleeFeb 14, 2022
  12. Elijah NewrenFeb 15, 2022
  13. Derrick StoleeFeb 15, 2022
  14. 6/7 sparse-checkout: reject arguments in cone-mode that look like patternsElijah Newren via GitGitGadget, Feb 13, 2022
  15. 5/7 sparse-checkout: reject non-cone-mode patterns starting with a '#'Elijah Newren via GitGitGadget, Feb 13, 2022
  16. Junio C HamanoFeb 14, 2022
  17. Elijah NewrenFeb 15, 2022
  18. Junio C HamanoFeb 16, 2022
  19. Elijah NewrenFeb 16, 2022
  20. Junio C HamanoFeb 16, 2022
  21. 7/7 sparse-checkout: make --cone the default and deprecate --no-coneElijah Newren via GitGitGadget, Feb 13, 2022
  22. Derrick StoleeFeb 14, 2022
  23. Elijah NewrenFeb 15, 2022
  24. Derrick StoleeFeb 14, 2022
  25. Elijah NewrenFeb 15, 2022
  26. Derrick StoleeFeb 15, 2022
  27. 0/6 sparse checkout: fix a few bugs and check argument validity for set/addElijah Newren via GitGitGadget, Feb 15, 2022
  28. 1/6 sparse-checkout: correct reapply's handling of optionsElijah Newren via GitGitGadget, Feb 15, 2022
  29. 2/6 sparse-checkout: correctly set non-cone mode when expectedElijah Newren via GitGitGadget, Feb 15, 2022
  30. 3/6 sparse-checkout: pay attention to prefix for {set, add}Elijah Newren via GitGitGadget, Feb 15, 2022
  31. Ævar Arnfjörð BjarmasonFeb 17, 2022
  32. Elijah NewrenFeb 18, 2022
  33. 4/6 sparse-checkout: error or warn when given individual filesElijah Newren via GitGitGadget, Feb 15, 2022
  34. Ævar Arnfjörð BjarmasonFeb 17, 2022
  35. 5/6 sparse-checkout: reject non-cone-mode patterns starting with a '#'Elijah Newren via GitGitGadget, Feb 15, 2022
  36. 6/6 sparse-checkout: reject arguments in cone-mode that look like patternsElijah Newren via GitGitGadget, Feb 15, 2022
  37. Derrick StoleeFeb 15, 2022
  38. 0/5 sparse checkout: fix a few bugs and check argument validity for set/addElijah Newren via GitGitGadget, Feb 16, 2022
  39. 2/5 sparse-checkout: correctly set non-cone mode when expectedElijah Newren via GitGitGadget, Feb 16, 2022
  40. 1/5 sparse-checkout: correct reapply's handling of optionsElijah Newren via GitGitGadget, Feb 16, 2022
  41. 3/5 sparse-checkout: pay attention to prefix for {set, add}Elijah Newren via GitGitGadget, Feb 16, 2022
  42. 4/5 sparse-checkout: error or warn when given individual filesElijah Newren via GitGitGadget, Feb 16, 2022
  43. 5/5 sparse-checkout: reject arguments in cone-mode that look like patternsElijah Newren via GitGitGadget, Feb 16, 2022
  44. Ævar Arnfjörð BjarmasonFeb 16, 2022
  45. Elijah NewrenFeb 16, 2022
  46. Victoria DyeFeb 16, 2022
  47. Junio C HamanoFeb 16, 2022
  48. Elijah NewrenFeb 17, 2022
  49. Junio C HamanoFeb 17, 2022
  50. Elijah NewrenFeb 17, 2022
  51. Elijah NewrenFeb 17, 2022
  52. Junio C HamanoFeb 16, 2022
  53. 0/5 sparse checkout: fix a few bugs and check argument validity for set/addElijah Newren via GitGitGadget, Feb 17, 2022
  54. 1/5 sparse-checkout: correct reapply's handling of optionsElijah Newren via GitGitGadget, Feb 17, 2022
  55. 2/5 sparse-checkout: correctly set non-cone mode when expectedElijah Newren via GitGitGadget, Feb 17, 2022
  56. 3/5 sparse-checkout: pay attention to prefix for {set, add}Elijah Newren via GitGitGadget, Feb 17, 2022
  57. Junio C HamanoFeb 17, 2022
  58. 5/5 sparse-checkout: reject arguments in cone-mode that look like patternsElijah Newren via GitGitGadget, Feb 17, 2022
  59. 4/5 sparse-checkout: error or warn when given individual filesElijah Newren via GitGitGadget, Feb 17, 2022
  60. Junio C HamanoFeb 17, 2022
  61. Elijah NewrenFeb 18, 2022
  62. Ævar Arnfjörð BjarmasonFeb 17, 2022
  63. 0/5 sparse checkout: fix a few bugs and check argument validity for set/addElijah Newren via GitGitGadget, Feb 19, 2022
  64. 1/5 sparse-checkout: correct reapply's handling of optionsElijah Newren via GitGitGadget, Feb 19, 2022
  65. 2/5 sparse-checkout: correctly set non-cone mode when expectedElijah Newren via GitGitGadget, Feb 19, 2022
  66. 3/5 sparse-checkout: pay attention to prefix for {set, add}Elijah Newren via GitGitGadget, Feb 19, 2022
  67. 4/5 sparse-checkout: error or warn when given individual filesElijah Newren via GitGitGadget, Feb 19, 2022
  68. 5/5 sparse-checkout: reject arguments in cone-mode that look like patternsElijah Newren via GitGitGadget, Feb 19, 2022
  69. Derrick StoleeFeb 20, 2022
  70. Junio C HamanoFeb 20, 2022

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.