git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v3 10/12] sparse-checkout: write escaped patterns in cone mode

From
Derrick Stolee <stolee@gmail.com>
Date
Jan 29, 2020, 14:16 UTC
Message-ID
<7a85cded-eccf-9f33-4056-08b2923b2861@gmail.com>
In-Reply-To
<20200129103352.GD4218@coredump.intra.peff.net>
On 1/29/2020 5:33 AM, Jeff King wrote:
Show 63 quoted lines
> On Wed, Jan 29, 2020 at 05:17:13AM -0500, Jeff King wrote:
> 
>>> From: Derrick Stolee <dstolee@microsoft.com>
>>>
>>> If a user somehow creates a directory with an asterisk (*) or backslash
>>> (\), then the "git sparse-checkout set" command will struggle to provide
>>> the correct pattern in the sparse-checkout file. When not in cone mode,
>>> the provided pattern is written directly into the sparse-checkout file.
>>> However, in cone mode we expect a list of paths to directories and then
>>> we convert those into patterns.
>>
>> Is this really about cone mode? It seems more like it is about --stdin.
>> I.e., what are the rules for when the input is a filename and when it is
>> a pattern? In our earlier discussion, I had assumed that command-line
>> arguments to "sparse-checkout set" were actual filenames, and "--stdin"
>> just read them from stdin.
>>
>> But looking at the documentation, they are always called "patterns" on
>> the command-line. Should the "--stdin" documentation make it clear that
>> we are no longer taking patterns, but instead actual filenames?
>>
>> Or am I confused, and in non-cone-mode the "ls-tree | sparse-checkout"
>> pipeline is not supposed to work at all? (I.e., they really are always
>> patterns)?
> 
> Hmph, sorry, I _was_ just confused. I was reading a copy of the manpage
> without your final patch, which made things much clearer.
> 
> So OK, I think the resulting documentation does make things clear. And
> this is just about cone mode, not --stdin. Please ignore my ramblings in
> the rest of the replied-to message. But...
> 
>>> Even more specifically, the goal is to always allow the following from
>>> the root of a repo:
>>>
>>>   git ls-tree --name-only -d HEAD | git sparse-checkout set --stdin
>>>
>>> The ls-tree command provides directory names with an unescaped asterisk.
>>> It also quotes the directories that contain an escaped backslash. We
>>> must remove these quotes, then keep the escaped backslashes.
>>>
>>> However, there is some care needed for the timing of these escapes. The
>>> in-memory pattern list is used to update the working directory before
>>> writing the patterns to disk. Thus, we need the command to have the
>>> unescaped names in the hashsets for the cone comparisons, then escape
>>> the patterns later.
>>
>> OK, this part make sense.
> 
> You could also demonstrate this even without --stdin with something
> like:
> 
>   git config core.sparsecheckoutcone true
>   git sparse-checkout set 'foo*bar'
> 
> which should take that as a literal filename and put the pattern
> 'foo\*bar' in the sparse-checkout file. And your tests do cover that.
> 
> So really there are two separate bugs here, and it might be a little
> easier to explain the "timing of these escapes" thing by doing them
> separately. I.e., the case above needs escaping and we could demonstrate
> the bug with a command-line "set".  And then follow up by fixing the
> problem with correctly de-quoting --stdin.

I've locally split the commit into two parts. That makes things much simpler to read.

Show 7 quoted lines
>>> +static char *escaped_pattern(char *pattern)
>> [...]
>> Do we need to catch other metacharacters here (using is_glob_special()
>> perhaps)?
> 
> After de-confusing myself, I think the individual code comments I wrote
> still apply though (especially this one).

I've applied the smaller comments and am now investigating the right thing to do with other is_glob_special() characters. There is a small chance that I can replace any "c == '*' || c == '\'" with is_glob_special(), but we shall see. At the very least, I'll need to expand my tests.

Thanks, -Stolee

Previous: Jeff KingNext: Derrick Stolee
Message 53 of 82 in “Harden the sparse-checkout builtin”
  1. 0/8 Harden the sparse-checkout builtinDerrick Stolee via GitGitGadget, Jan 14, 2020
  2. 2/8 sparse-checkout: create leading directoriesDerrick Stolee via GitGitGadget, Jan 14, 2020
  3. Junio C HamanoJan 16, 2020
  4. 1/8 t1091: use check_files to reduce boilerplateDerrick Stolee via GitGitGadget, Jan 14, 2020
  5. Junio C HamanoJan 16, 2020
  6. 4/8 sparse-checkout: cone mode does not recognize "**"Derrick Stolee via GitGitGadget, Jan 14, 2020
  7. Jeff KingJan 14, 2020
  8. 5/8 sparse-checkout: detect short patternsDerrick Stolee via GitGitGadget, Jan 14, 2020
  9. 3/8 clone: fix --sparse option with URLsDerrick Stolee via GitGitGadget, Jan 14, 2020
  10. Taylor BlauJan 14, 2020
  11. 7/8 sparse-checkout: properly match escaped charactersDerrick Stolee via GitGitGadget, Jan 14, 2020
  12. Jeff KingJan 14, 2020
  13. Derrick StoleeJan 14, 2020
  14. 6/8 sparse-checkout: warn on incorrect '*' in patternsDerrick Stolee via GitGitGadget, Jan 14, 2020
  15. 8/8 sparse-checkout: write escaped patterns in cone modeDerrick Stolee via GitGitGadget, Jan 14, 2020
  16. Jeff KingJan 14, 2020
  17. Derrick StoleeJan 14, 2020
  18. Jeff KingJan 14, 2020
  19. Derrick StoleeJan 24, 2020
  20. Jeff KingJan 24, 2020
  21. Derrick StoleeJan 28, 2020
  22. Taylor BlauJan 14, 2020
  23. Derrick StoleeJan 14, 2020
  24. Jeff KingJan 14, 2020
  25. Junio C HamanoJan 15, 2020
  26. Derrick StoleeJan 15, 2020
  27. 00/12 Harden the sparse-checkout builtinDerrick Stolee via GitGitGadget, Jan 24, 2020
  28. 01/12 t1091: use check_files to reduce boilerplateDerrick Stolee via GitGitGadget, Jan 24, 2020
  29. 02/12 t1091: improve here-docsDerrick Stolee via GitGitGadget, Jan 24, 2020
  30. 04/12 clone: fix --sparse option with URLsDerrick Stolee via GitGitGadget, Jan 24, 2020
  31. 05/12 sparse-checkout: fix documentation typo for core.sparseCheckoutConeJeff King via GitGitGadget, Jan 24, 2020
  32. 06/12 sparse-checkout: cone mode does not recognize "**"Derrick Stolee via GitGitGadget, Jan 24, 2020
  33. 03/12 sparse-checkout: create leading directoriesDerrick Stolee via GitGitGadget, Jan 24, 2020
  34. 09/12 sparse-checkout: properly match escaped charactersDerrick Stolee via GitGitGadget, Jan 24, 2020
  35. 08/12 sparse-checkout: warn on incorrect '*' in patternsDerrick Stolee via GitGitGadget, Jan 24, 2020
  36. 07/12 sparse-checkout: detect short patternsDerrick Stolee via GitGitGadget, Jan 24, 2020
  37. 12/12 sparse-checkout: improve docs around 'set' in cone modeDerrick Stolee via GitGitGadget, Jan 24, 2020
  38. 10/12 sparse-checkout: write escaped patterns in cone modeDerrick Stolee via GitGitGadget, Jan 24, 2020
  39. 11/12 sparse-checkout: use C-style quotes in 'list' subcommandDerrick Stolee via GitGitGadget, Jan 24, 2020
  40. 00/12 Harden the sparse-checkout builtinDerrick Stolee via GitGitGadget, Jan 28, 2020
  41. 01/12 t1091: use check_files to reduce boilerplateDerrick Stolee via GitGitGadget, Jan 28, 2020
  42. 02/12 t1091: improve here-docsDerrick Stolee via GitGitGadget, Jan 28, 2020
  43. 04/12 clone: fix --sparse option with URLsDerrick Stolee via GitGitGadget, Jan 28, 2020
  44. 05/12 sparse-checkout: fix documentation typo for core.sparseCheckoutConeJeff King via GitGitGadget, Jan 28, 2020
  45. 03/12 sparse-checkout: create leading directoriesDerrick Stolee via GitGitGadget, Jan 28, 2020
  46. 06/12 sparse-checkout: cone mode does not recognize "**"Derrick Stolee via GitGitGadget, Jan 28, 2020
  47. 11/12 sparse-checkout: use C-style quotes in 'list' subcommandDerrick Stolee via GitGitGadget, Jan 28, 2020
  48. Jeff KingJan 29, 2020
  49. 12/12 sparse-checkout: improve docs around 'set' in cone modeDerrick Stolee via GitGitGadget, Jan 28, 2020
  50. 10/12 sparse-checkout: write escaped patterns in cone modeDerrick Stolee via GitGitGadget, Jan 28, 2020
  51. Jeff KingJan 29, 2020
  52. Jeff KingJan 29, 2020
  53. Derrick StoleeJan 29, 2020
  54. Derrick StoleeJan 29, 2020
  55. Jeff KingJan 30, 2020
  56. Derrick StoleeJan 30, 2020
  57. 08/12 sparse-checkout: warn on incorrect '*' in patternsDerrick Stolee via GitGitGadget, Jan 28, 2020
  58. 07/12 sparse-checkout: detect short patternsDerrick Stolee via GitGitGadget, Jan 28, 2020
  59. 09/12 sparse-checkout: properly match escaped charactersDerrick Stolee via GitGitGadget, Jan 28, 2020
  60. Jeff KingJan 29, 2020
  61. Derrick StoleeJan 29, 2020
  62. Derrick StoleeJan 29, 2020
  63. 00/15 Harden the sparse-checkout builtinDerrick Stolee via GitGitGadget, Jan 31, 2020
  64. 01/15 t1091: use check_files to reduce boilerplateDerrick Stolee via GitGitGadget, Jan 31, 2020
  65. 02/15 t1091: improve here-docsDerrick Stolee via GitGitGadget, Jan 31, 2020
  66. 04/15 clone: fix --sparse option with URLsDerrick Stolee via GitGitGadget, Jan 31, 2020
  67. 06/15 sparse-checkout: cone mode does not recognize "**"Derrick Stolee via GitGitGadget, Jan 31, 2020
  68. 07/15 sparse-checkout: detect short patternsDerrick Stolee via GitGitGadget, Jan 31, 2020
  69. 05/15 sparse-checkout: fix documentation typo for core.sparseCheckoutConeJeff King via GitGitGadget, Jan 31, 2020
  70. 08/15 sparse-checkout: warn on globs in cone patternsDerrick Stolee via GitGitGadget, Jan 31, 2020
  71. 03/15 sparse-checkout: create leading directoriesDerrick Stolee via GitGitGadget, Jan 31, 2020
  72. 09/15 sparse-checkout: properly match escaped charactersDerrick Stolee via GitGitGadget, Jan 31, 2020
  73. 11/15 sparse-checkout: unquote C-style strings over --stdinDerrick Stolee via GitGitGadget, Jan 31, 2020
  74. 10/15 sparse-checkout: write escaped patterns in cone modeDerrick Stolee via GitGitGadget, Jan 31, 2020
  75. 15/15 sparse-checkout: fix cone mode behavior mismatchDerrick Stolee via GitGitGadget, Jan 31, 2020
  76. 14/15 sparse-checkout: improve docs around 'set' in cone modeDerrick Stolee via GitGitGadget, Jan 31, 2020
  77. 12/15 sparse-checkout: use C-style quotes in 'list' subcommandDerrick Stolee via GitGitGadget, Jan 31, 2020
  78. 13/15 sparse-checkout: escape all glob characters on writeDerrick Stolee via GitGitGadget, Jan 31, 2020
  79. Elijah NewrenJan 31, 2020
  80. Derrick StoleeFeb 3, 2020
  81. Taylor BlauFeb 8, 2020
  82. Junio C HamanoFeb 9, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.