git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 10/12] sparse-checkout: write escaped patterns in cone mode

From
Derrick Stolee via GitGitGadget <gitgitgadget@gmail.com>
Date
Jan 28, 2020, 18:26 UTC
Message-ID
<e2f9afc70c92c606a9d93660bce0599b7b7ddf6e.1580236003.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.513.v3.git.1580236003.gitgitgadget@gmail.com>
From: Derrick Stolee <dstolee@microsoft.com>

If a user somehow creates a directory with an asterisk (*) or backslash (\), then the "git sparse-checkout set" command will struggle to provide the correct pattern in the sparse-checkout file. When not in cone mode, the provided pattern is written directly into the sparse-checkout file. However, in cone mode we expect a list of paths to directories and then we convert those into patterns.

Even more specifically, the goal is to always allow the following from the root of a repo:

  git ls-tree --name-only -d HEAD | git sparse-checkout set --stdin

The ls-tree command provides directory names with an unescaped asterisk. It also quotes the directories that contain an escaped backslash. We must remove these quotes, then keep the escaped backslashes.

However, there is some care needed for the timing of these escapes. The in-memory pattern list is used to update the working directory before writing the patterns to disk. Thus, we need the command to have the unescaped names in the hashsets for the cone comparisons, then escape the patterns later.

Use unquote_c_style() when parsing lines from stdin. Command-line arguments will be parsed as-is, assuming the user can do the correct level of escaping from their environment to match the exact directory names.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
 builtin/sparse-checkout.c          | 38 +++++++++++++++++++++++++++---
 t/t1091-sparse-checkout-builtin.sh | 21 +++++++++++++++--
 2 files changed, 54 insertions(+), 5 deletions(-)
diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c
index 3cee8ab46e..61414fef18 100644
--- a/builtin/sparse-checkout.c
+++ b/builtin/sparse-checkout.c
@@ -13,6 +13,7 @@
 #include "resolve-undo.h"
 #include "unpack-trees.h"
 #include "wt-status.h"
+#include "quote.h"
 
 static const char *empty_base = "";
 
@@ -140,6 +141,22 @@ static int update_working_directory(struct pattern_list *pl)
 	return result;
 }
 
+static char *escaped_pattern(char *pattern)
+{
+	char *p = pattern;
+	struct strbuf final = STRBUF_INIT;
+
+	while (*p) {
+		if (*p == '*' || *p == '\\')
+			strbuf_addch(&final, '\\');
+
+		strbuf_addch(&final, *p);
+		p++;
+	}
+
+	return strbuf_detach(&final, NULL);
+}
+
 static void write_cone_to_file(FILE *fp, struct pattern_list *pl)
 {
 	int i;
@@ -164,10 +181,11 @@ static void write_cone_to_file(FILE *fp, struct pattern_list *pl)
 	fprintf(fp, "/*\n!/*/\n");
 
 	for (i = 0; i < sl.nr; i++) {
-		char *pattern = sl.items[i].string;
+		char *pattern = escaped_pattern(sl.items[i].string);
 
 		if (strlen(pattern))
 			fprintf(fp, "%s/\n!%s/*/\n", pattern, pattern);
+		free(pattern);
 	}
 
 	string_list_clear(&sl, 0);
@@ -185,8 +203,9 @@ static void write_cone_to_file(FILE *fp, struct pattern_list *pl)
 	string_list_remove_duplicates(&sl, 0);
 
 	for (i = 0; i < sl.nr; i++) {
-		char *pattern = sl.items[i].string;
+		char *pattern = escaped_pattern(sl.items[i].string);
 		fprintf(fp, "%s/\n", pattern);
+		free(pattern);
 	}
 }
 
@@ -423,8 +442,21 @@ static int sparse_checkout_set(int argc, const char **argv, const char *prefix)
 		pl.use_cone_patterns = 1;
 
 		if (set_opts.use_stdin) {
-			while (!strbuf_getline(&line, stdin))
+			struct strbuf unquoted = STRBUF_INIT;
+			while (!strbuf_getline(&line, stdin)) {
+				if (line.buf[0] == '"') {
+					strbuf_setlen(&unquoted, 0);
+					if (unquote_c_style(&unquoted, line.buf, NULL))
+						die(_("unable to unquote C-style string '%s'"),
+						line.buf);
+
+					strbuf_swap(&unquoted, &line);
+				}
+
 				strbuf_to_cone_pattern(&line, &pl);
+			}
+
+			strbuf_release(&unquoted);
 		} else {
 			for (i = 0; i < argc; i++) {
 				strbuf_setlen(&line, 0);
diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index 0a21a5e15d..459715d541 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -309,6 +309,9 @@ check_read_tree_errors () {
 	REPO=$1
 	FILES=$2
 	ERRORS=$3
+	git -C $REPO -c core.sparseCheckoutCone=false read-tree -mu HEAD 2>err &&
+	test_must_be_empty err &&
+	check_files $REPO "$FILES" &&
 	git -C $REPO read-tree -mu HEAD 2>err &&
 	if test -z "$ERRORS"
 	then
@@ -379,14 +382,28 @@ test_expect_success BSLASHPSPEC 'pattern-checks: escaped "*"' '
 	git -C escaped reset --hard $COMMIT &&
 	check_files escaped "a deep folder1 folder2 zbad\\dir zdoes*exist" &&
 	git -C escaped sparse-checkout init --cone &&
-	cat >escaped/.git/info/sparse-checkout <<-\EOF &&
+	git -C escaped sparse-checkout set zbad\\dir "zdoes*not*exist" "zdoes*exist" &&
+	cat >expect <<-\EOF &&
 	/*
 	!/*/
 	/zbad\\dir/
+	/zdoes\*exist/
 	/zdoes\*not\*exist/
+	EOF
+	test_cmp expect escaped/.git/info/sparse-checkout &&
+	check_read_tree_errors escaped "a zbad\\dir zdoes*exist" &&
+	git -C escaped ls-tree -d --name-only HEAD | git -C escaped sparse-checkout set --stdin &&
+	cat >expect <<-\EOF &&
+	/*
+	!/*/
+	/deep/
+	/folder1/
+	/folder2/
+	/zbad\\dir/
 	/zdoes\*exist/
 	EOF
-	check_read_tree_errors escaped "a zbad\\dir zdoes*exist"
+	test_cmp expect escaped/.git/info/sparse-checkout &&
+	check_files escaped "a deep folder1 folder2 zbad\\dir zdoes*exist"
 '
 
 test_done
-- 
gitgitgadget
Previous: Derrick Stolee via GitGitGadgetNext: Jeff King
Message 50 of 82 in “Harden the sparse-checkout builtin”
  1. 0/8 Harden the sparse-checkout builtinDerrick Stolee via GitGitGadget, Jan 14, 2020
  2. 2/8 sparse-checkout: create leading directoriesDerrick Stolee via GitGitGadget, Jan 14, 2020
  3. Junio C HamanoJan 16, 2020
  4. 1/8 t1091: use check_files to reduce boilerplateDerrick Stolee via GitGitGadget, Jan 14, 2020
  5. Junio C HamanoJan 16, 2020
  6. 4/8 sparse-checkout: cone mode does not recognize "**"Derrick Stolee via GitGitGadget, Jan 14, 2020
  7. Jeff KingJan 14, 2020
  8. 5/8 sparse-checkout: detect short patternsDerrick Stolee via GitGitGadget, Jan 14, 2020
  9. 3/8 clone: fix --sparse option with URLsDerrick Stolee via GitGitGadget, Jan 14, 2020
  10. Taylor BlauJan 14, 2020
  11. 7/8 sparse-checkout: properly match escaped charactersDerrick Stolee via GitGitGadget, Jan 14, 2020
  12. Jeff KingJan 14, 2020
  13. Derrick StoleeJan 14, 2020
  14. 6/8 sparse-checkout: warn on incorrect '*' in patternsDerrick Stolee via GitGitGadget, Jan 14, 2020
  15. 8/8 sparse-checkout: write escaped patterns in cone modeDerrick Stolee via GitGitGadget, Jan 14, 2020
  16. Jeff KingJan 14, 2020
  17. Derrick StoleeJan 14, 2020
  18. Jeff KingJan 14, 2020
  19. Derrick StoleeJan 24, 2020
  20. Jeff KingJan 24, 2020
  21. Derrick StoleeJan 28, 2020
  22. Taylor BlauJan 14, 2020
  23. Derrick StoleeJan 14, 2020
  24. Jeff KingJan 14, 2020
  25. Junio C HamanoJan 15, 2020
  26. Derrick StoleeJan 15, 2020
  27. 00/12 Harden the sparse-checkout builtinDerrick Stolee via GitGitGadget, Jan 24, 2020
  28. 01/12 t1091: use check_files to reduce boilerplateDerrick Stolee via GitGitGadget, Jan 24, 2020
  29. 02/12 t1091: improve here-docsDerrick Stolee via GitGitGadget, Jan 24, 2020
  30. 04/12 clone: fix --sparse option with URLsDerrick Stolee via GitGitGadget, Jan 24, 2020
  31. 05/12 sparse-checkout: fix documentation typo for core.sparseCheckoutConeJeff King via GitGitGadget, Jan 24, 2020
  32. 06/12 sparse-checkout: cone mode does not recognize "**"Derrick Stolee via GitGitGadget, Jan 24, 2020
  33. 03/12 sparse-checkout: create leading directoriesDerrick Stolee via GitGitGadget, Jan 24, 2020
  34. 09/12 sparse-checkout: properly match escaped charactersDerrick Stolee via GitGitGadget, Jan 24, 2020
  35. 08/12 sparse-checkout: warn on incorrect '*' in patternsDerrick Stolee via GitGitGadget, Jan 24, 2020
  36. 07/12 sparse-checkout: detect short patternsDerrick Stolee via GitGitGadget, Jan 24, 2020
  37. 12/12 sparse-checkout: improve docs around 'set' in cone modeDerrick Stolee via GitGitGadget, Jan 24, 2020
  38. 10/12 sparse-checkout: write escaped patterns in cone modeDerrick Stolee via GitGitGadget, Jan 24, 2020
  39. 11/12 sparse-checkout: use C-style quotes in 'list' subcommandDerrick Stolee via GitGitGadget, Jan 24, 2020
  40. 00/12 Harden the sparse-checkout builtinDerrick Stolee via GitGitGadget, Jan 28, 2020
  41. 01/12 t1091: use check_files to reduce boilerplateDerrick Stolee via GitGitGadget, Jan 28, 2020
  42. 02/12 t1091: improve here-docsDerrick Stolee via GitGitGadget, Jan 28, 2020
  43. 04/12 clone: fix --sparse option with URLsDerrick Stolee via GitGitGadget, Jan 28, 2020
  44. 05/12 sparse-checkout: fix documentation typo for core.sparseCheckoutConeJeff King via GitGitGadget, Jan 28, 2020
  45. 03/12 sparse-checkout: create leading directoriesDerrick Stolee via GitGitGadget, Jan 28, 2020
  46. 06/12 sparse-checkout: cone mode does not recognize "**"Derrick Stolee via GitGitGadget, Jan 28, 2020
  47. 11/12 sparse-checkout: use C-style quotes in 'list' subcommandDerrick Stolee via GitGitGadget, Jan 28, 2020
  48. Jeff KingJan 29, 2020
  49. 12/12 sparse-checkout: improve docs around 'set' in cone modeDerrick Stolee via GitGitGadget, Jan 28, 2020
  50. 10/12 sparse-checkout: write escaped patterns in cone modeDerrick Stolee via GitGitGadget, Jan 28, 2020
  51. Jeff KingJan 29, 2020
  52. Jeff KingJan 29, 2020
  53. Derrick StoleeJan 29, 2020
  54. Derrick StoleeJan 29, 2020
  55. Jeff KingJan 30, 2020
  56. Derrick StoleeJan 30, 2020
  57. 08/12 sparse-checkout: warn on incorrect '*' in patternsDerrick Stolee via GitGitGadget, Jan 28, 2020
  58. 07/12 sparse-checkout: detect short patternsDerrick Stolee via GitGitGadget, Jan 28, 2020
  59. 09/12 sparse-checkout: properly match escaped charactersDerrick Stolee via GitGitGadget, Jan 28, 2020
  60. Jeff KingJan 29, 2020
  61. Derrick StoleeJan 29, 2020
  62. Derrick StoleeJan 29, 2020
  63. 00/15 Harden the sparse-checkout builtinDerrick Stolee via GitGitGadget, Jan 31, 2020
  64. 01/15 t1091: use check_files to reduce boilerplateDerrick Stolee via GitGitGadget, Jan 31, 2020
  65. 02/15 t1091: improve here-docsDerrick Stolee via GitGitGadget, Jan 31, 2020
  66. 04/15 clone: fix --sparse option with URLsDerrick Stolee via GitGitGadget, Jan 31, 2020
  67. 06/15 sparse-checkout: cone mode does not recognize "**"Derrick Stolee via GitGitGadget, Jan 31, 2020
  68. 07/15 sparse-checkout: detect short patternsDerrick Stolee via GitGitGadget, Jan 31, 2020
  69. 05/15 sparse-checkout: fix documentation typo for core.sparseCheckoutConeJeff King via GitGitGadget, Jan 31, 2020
  70. 08/15 sparse-checkout: warn on globs in cone patternsDerrick Stolee via GitGitGadget, Jan 31, 2020
  71. 03/15 sparse-checkout: create leading directoriesDerrick Stolee via GitGitGadget, Jan 31, 2020
  72. 09/15 sparse-checkout: properly match escaped charactersDerrick Stolee via GitGitGadget, Jan 31, 2020
  73. 11/15 sparse-checkout: unquote C-style strings over --stdinDerrick Stolee via GitGitGadget, Jan 31, 2020
  74. 10/15 sparse-checkout: write escaped patterns in cone modeDerrick Stolee via GitGitGadget, Jan 31, 2020
  75. 15/15 sparse-checkout: fix cone mode behavior mismatchDerrick Stolee via GitGitGadget, Jan 31, 2020
  76. 14/15 sparse-checkout: improve docs around 'set' in cone modeDerrick Stolee via GitGitGadget, Jan 31, 2020
  77. 12/15 sparse-checkout: use C-style quotes in 'list' subcommandDerrick Stolee via GitGitGadget, Jan 31, 2020
  78. 13/15 sparse-checkout: escape all glob characters on writeDerrick Stolee via GitGitGadget, Jan 31, 2020
  79. Elijah NewrenJan 31, 2020
  80. Derrick StoleeFeb 3, 2020
  81. Taylor BlauFeb 8, 2020
  82. Junio C HamanoFeb 9, 2020

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.