{"thread":{"id":"44147","subject":"[PATCH 0/3] recursive support for ls-files","startedAt":"2016-09-24T00:13:56Z","lastAt":"2016-10-07T18:45:11Z","messageCount":67,"participants":["Brandon Williams","Jeff King","Junio C Hamano","Stefan Beller"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"302472","messageId":"1474676014-134568-1-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":null,"subject":"[PATCH 0/3] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-24T00:13:31Z","receivedAt":"2016-09-24T00:13:56Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"After looking at the feedback I rerolled a few things, in particular the\n--submodule_prefix option that existed to give a submodule context about where\nit had been invoked from. People didn't seem to like the idea of exposing this\nto the users (yet anyways) so I removed it as an option and instead have it\nbeing passed to a child process via an environment variable\nGIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to\nexternal users at the moment.\n\nAlso fixed a bug (and added a test) for the -z options as pointed out by Jeff\nKing.\n\nBrandon Williams (3):\n submodules: make submodule-prefix option an envvar\n ls-files: optionally recurse into submodules\n ls-files: add pathspec matching for submodules\n\n Documentation/git-ls-files.txt | 7 +-\n builtin/ls-files.c | 173 ++++++++++++++++++++-------\n cache.h | 1 +\n dir.c | 46 +++++++-\n dir.h | 4 +\n environment.c | 1 +\n t/t3007-ls-files-recurse-submodules.sh | 209 +++++++++++++++++++++++++++++++++\n 7 files changed, 398 insertions(+), 43 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302473","messageId":"1474676014-134568-2-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474676014-134568-1-git-send-email-bmwill@google.com","subject":"[PATCH 1/3 v3] submodules: make submodule-prefix option an envvar","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-24T00:13:32Z","receivedAt":"2016-09-24T00:14:00Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add a submodule-prefix enviorment variable\n'GIT_INTERNAL_SUBMODULE_PREFIX' which can be used by commands which have\n--recurse-submodule options.\n\nSigned-off-by: Brandon Williams \n---\n cache.h | 1 +\n environment.c | 1 +\n 2 files changed, 2 insertions(+)\n\ndiff --git a/cache.h b/cache.h\nindex 3556326..ae88a35 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)\n #define GIT_NAMESPACE_ENVIRONMENT \"GIT_NAMESPACE\"\n #define GIT_WORK_TREE_ENVIRONMENT \"GIT_WORK_TREE\"\n #define GIT_PREFIX_ENVIRONMENT \"GIT_PREFIX\"\n+#define GIT_SUBMODULE_PREFIX_ENVIRONMENT \"GIT_INTERNAL_SUBMODULE_PREFIX\"\n #define DEFAULT_GIT_DIR_ENVIRONMENT \".git\"\n #define DB_ENVIRONMENT \"GIT_OBJECT_DIRECTORY\"\n #define INDEX_ENVIRONMENT \"GIT_INDEX_FILE\"\ndiff --git a/environment.c b/environment.c\nindex ca72464..7380815 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -120,6 +120,7 @@ const char * const local_repo_env[] = {\n \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n \tGIT_PREFIX_ENVIRONMENT,\n+\tGIT_SUBMODULE_PREFIX_ENVIRONMENT,\n \tGIT_SHALLOW_FILE_ENVIRONMENT,\n \tGIT_COMMON_DIR_ENVIRONMENT,\n \tNULL\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302474","messageId":"1474676014-134568-3-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474676014-134568-1-git-send-email-bmwill@google.com","subject":"[PATCH 2/3 v3] ls-files: optionally recurse into submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-24T00:13:33Z","receivedAt":"2016-09-24T00:14:06Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Allow ls-files to recognize submodules in order to retrieve a list of\nfiles from a repository's submodules. This is done by forking off a\nprocess to recursively call ls-files on all submodules. Use environment\nvariable `GIT_INTERNAL_SUBMODULE_PREFIX` to pass a path to the submodule\nwhich it can use to prepend to output or pathspec matching logic.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git-ls-files.txt | 7 ++-\n builtin/ls-files.c | 63 ++++++++++++++++++++++\n t/t3007-ls-files-recurse-submodules.sh | 99 ++++++++++++++++++++++++++++++++++\n 3 files changed, 168 insertions(+), 1 deletion(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\ndiff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\nindex 0d933ac..446209e 100644\n--- a/Documentation/git-ls-files.txt\n+++ b/Documentation/git-ls-files.txt\n@@ -18,7 +18,8 @@ SYNOPSIS\n \t\t[--exclude-per-directory=]\n \t\t[--exclude-standard]\n \t\t[--error-unmatch] [--with-tree=]\n-\t\t[--full-name] [--abbrev] [--] [...]\n+\t\t[--full-name] [--recurse-submodules]\n+\t\t[--abbrev] [--] [...]\n \n DESCRIPTION\n -----------\n@@ -137,6 +138,10 @@ a space) at the start of each line:\n \toption forces paths to be output relative to the project\n \ttop directory.\n \n+--recurse-submodules::\n+\tRecursively calls ls-files on each submodule in the repository.\n+\tCurrently there is only support for the --cached mode.\n+\n --abbrev[=]::\n \tInstead of showing the full 40-byte hexadecimal object\n \tlines, show only a partial prefix.\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 00ea91a..54ab765 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -14,6 +14,7 @@\n #include \"resolve-undo.h\"\n #include \"string-list.h\"\n #include \"pathspec.h\"\n+#include \"run-command.h\"\n \n static int abbrev;\n static int show_deleted;\n@@ -28,6 +29,8 @@ static int show_valid_bit;\n static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n+static int recurse_submodules;\n+static const char *submodule_prefix;\n \n static const char *prefix;\n static int max_prefix_len;\n@@ -68,6 +71,21 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)\n static void write_name(const char *name)\n {\n \t/*\n+\t * NEEDSWORK: To make this thread-safe, full_name would have to be owned\n+\t * by the caller.\n+\t *\n+\t * full_name get reused across output lines to minimize the allocation\n+\t * churn.\n+\t */\n+\tstatic struct strbuf full_name = STRBUF_INIT;\n+\tif (submodule_prefix && *submodule_prefix) {\n+\t\tstrbuf_reset(&full_name);\n+\t\tstrbuf_addstr(&full_name, submodule_prefix);\n+\t\tstrbuf_addstr(&full_name, name);\n+\t\tname = full_name.buf;\n+\t}\n+\n+\t/*\n \t * With \"--full-name\", prefix_len=0; this caller needs to pass\n \t * an empty string in that case (a NULL is good for \"\").\n \t */\n@@ -152,6 +170,27 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/**\n+ * Recursively call ls-files on a submodule\n+ */\n+static void show_gitlink(const struct cache_entry *ce)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tint status;\n+\n+\targv_array_push(&cp.args, \"ls-files\");\n+\targv_array_push(&cp.args, \"--recurse-submodules\");\n+\targv_array_pushf(&cp.env_array, \"%s=%s%s/\",\n+\t\t\t GIT_SUBMODULE_PREFIX_ENVIRONMENT,\n+\t\t\t submodule_prefix ? submodule_prefix : \"\",\n+\t\t\t ce->name);\n+\tcp.git_cmd = 1;\n+\tcp.dir = ce->name;\n+\tstatus = run_command(&cp);\n+\tif (status)\n+\t\texit(status);\n+}\n+\n static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n {\n \tint len = max_prefix_len;\n@@ -163,6 +202,10 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n \t\t\t len, ps_matched,\n \t\t\t S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)))\n \t\treturn;\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\t\tshow_gitlink(ce);\n+\t\treturn;\n+\t}\n \n \tif (tag && *tag && show_valid_bit &&\n \t (ce->ce_flags & CE_VALID)) {\n@@ -468,6 +511,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\t{ OPTION_SET_INT, 0, \"full-name\", &prefix_len, NULL,\n \t\t\tN_(\"make the output relative to the project top directory\"),\n \t\t\tPARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL },\n+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\tN_(\"recurse through submodules\")),\n \t\tOPT_BOOL(0, \"error-unmatch\", &error_unmatch,\n \t\t\tN_(\"if any is not in the index, treat this as an error\")),\n \t\tOPT_STRING(0, \"with-tree\", &with_tree, N_(\"tree-ish\"),\n@@ -519,6 +564,24 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules)\n+\t\tsubmodule_prefix = getenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT);\n+\n+\tif (recurse_submodules &&\n+\t (show_stage || show_deleted || show_others || show_unmerged ||\n+\t show_killed || show_modified || show_resolve_undo ||\n+\t show_valid_bit || show_tag || show_eol))\n+\t\tdie(\"ls-files --recurse-submodules can only be used in \"\n+\t\t \"--cached mode\");\n+\n+\tif (recurse_submodules && error_unmatch)\n+\t\tdie(\"ls-files --recurse-submodules does not support \"\n+\t\t \"--error-unmatch\");\n+\n+\tif (recurse_submodules && argc)\n+\t\tdie(\"ls-files --recurse-submodules does not support path \"\n+\t\t \"arguments\");\n+\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nnew file mode 100755\nindex 0000000..caf3815\n--- /dev/null\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -0,0 +1,99 @@\n+#!/bin/sh\n+\n+test_description='Test ls-files recurse-submodules feature\n+\n+This test verifies the recurse-submodules feature correctly lists files from\n+submodules.\n+'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup directory structure and submodules' '\n+\techo a >a &&\n+\tmkdir b &&\n+\techo b >b/b &&\n+\tgit add a b &&\n+\tgit commit -m \"add a and b\" &&\n+\tgit init submodule &&\n+\techo c >submodule/c &&\n+\tgit -C submodule add c &&\n+\tgit -C submodule commit -m \"add c\" &&\n+\tgit submodule add ./submodule &&\n+\tgit commit -m \"added submodule\"\n+'\n+\n+test_expect_success 'ls-files correctly outputs files in submodule' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files does not output files not added to a repo' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\techo a >not_added &&\n+\techo b >b/not_added &&\n+\techo c >submodule/not_added &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files recurses more than 1 level' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/subsub/d\n+\tEOF\n+\n+\tgit init submodule/subsub &&\n+\techo d >submodule/subsub/d &&\n+\tgit -C submodule/subsub add d &&\n+\tgit -C submodule/subsub commit -m \"add d\" &&\n+\tgit -C submodule submodule add ./subsub &&\n+\tgit -C submodule commit -m \"added subsub\" &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support using path arguments' '\n+\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n+\ttest_i18ngrep \"does not support path arguments\" actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support --error-unmatch' '\n+\ttest_must_fail git ls-files --recurse-submodules --error-unmatch 2>actual &&\n+\ttest_i18ngrep \"does not support --error-unmatch\" actual\n+'\n+\n+test_incompatible_with_recurse_submodules () {\n+\ttest_expect_success \"--recurse-submodules and $1 are incompatible\" \"\n+\t\ttest_must_fail git ls-files --recurse-submodules $1 2>actual &&\n+\t\ttest_i18ngrep 'can only be used in --cached mode' actual\n+\t\"\n+}\n+\n+test_incompatible_with_recurse_submodules -v\n+test_incompatible_with_recurse_submodules -t\n+test_incompatible_with_recurse_submodules --deleted\n+test_incompatible_with_recurse_submodules --modified\n+test_incompatible_with_recurse_submodules --others\n+test_incompatible_with_recurse_submodules --stage\n+test_incompatible_with_recurse_submodules --killed\n+test_incompatible_with_recurse_submodules --unmerged\n+test_incompatible_with_recurse_submodules --eol\n+\n+test_done\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302475","messageId":"1474676014-134568-4-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474676014-134568-1-git-send-email-bmwill@google.com","subject":"[PATCH 3/3 v3] ls-files: add pathspec matching for submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-24T00:13:34Z","receivedAt":"2016-09-24T00:14:09Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pathspecs can be a bit tricky when trying to apply them to submodules.\nThe main challenge is that the pathspecs will be with respect to the\nsuperproject and not with respect to paths in the submodule. The\napproach this patch takes is to pass in the identical pathspec from the\nsuperproject to the submodule in addition to the submodule-prefix, which\nis the path from the root of the superproject to the submodule, and then\nwe can compare an entry in the submodule prepended with the\nsubmodule-prefix to the pathspec in order to determine if there is a\nmatch.\n\nThis patch also permits the pathspec logic to perform a prefix match against\nsubmodules since a pathspec could refer to a file inside of a submodule.\nDue to limitations in the wildmatch logic, a prefix match is only done\nliterally. If any wildcard character is encountered we'll simply punt\nand produce a false positive match. More accurate matching will be done\nonce inside the submodule. This is due to the superproject not knowing\nwhat files could exist in the submodule.\n\nSigned-off-by: Brandon Williams \n---\n builtin/ls-files.c | 134 ++++++++++++++++++++-------------\n dir.c | 46 ++++++++++-\n dir.h | 4 +\n t/t3007-ls-files-recurse-submodules.sh | 126 +++++++++++++++++++++++++++++--\n 4 files changed, 248 insertions(+), 62 deletions(-)\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 54ab765..8ecffd1 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -177,6 +177,7 @@ static void show_gitlink(const struct cache_entry *ce)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tint status;\n+\tint i;\n \n \targv_array_push(&cp.args, \"ls-files\");\n \targv_array_push(&cp.args, \"--recurse-submodules\");\n@@ -184,6 +185,29 @@ static void show_gitlink(const struct cache_entry *ce)\n \t\t\t GIT_SUBMODULE_PREFIX_ENVIRONMENT,\n \t\t\t submodule_prefix ? submodule_prefix : \"\",\n \t\t\t ce->name);\n+\t/* add options */\n+\tif (show_eol)\n+\t\targv_array_push(&cp.args, \"--eol\");\n+\tif (show_valid_bit)\n+\t\targv_array_push(&cp.args, \"-v\");\n+\tif (show_stage)\n+\t\targv_array_push(&cp.args, \"--stage\");\n+\tif (show_cached)\n+\t\targv_array_push(&cp.args, \"--cached\");\n+\tif (debug_mode)\n+\t\targv_array_push(&cp.args, \"--debug\");\n+\tif (line_terminator == '\\0')\n+\t\targv_array_push(&cp.args, \"-z\");\n+\n+\t/*\n+\t * Pass in the original pathspec args. The submodule will be\n+\t * responsible for prepending the 'submodule_prefix' prior to comparing\n+\t * against the pathspec for matches.\n+\t */\n+\targv_array_push(&cp.args, \"--\");\n+\tfor (i = 0; i < pathspec.nr; i++)\n+\t\targv_array_push(&cp.args, pathspec.items[i].original);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -193,57 +217,62 @@ static void show_gitlink(const struct cache_entry *ce)\n \n static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n {\n+\tstruct strbuf name = STRBUF_INIT;\n \tint len = max_prefix_len;\n+\tif (submodule_prefix)\n+\t\tstrbuf_addstr(&name, submodule_prefix);\n+\tstrbuf_addstr(&name, ce->name);\n \n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (!match_pathspec(&pathspec, ce->name, ce_namelen(ce),\n-\t\t\t len, ps_matched,\n-\t\t\t S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)))\n-\t\treturn;\n-\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&\n+\t submodule_path_match(&pathspec, name.buf, ps_matched)) {\n \t\tshow_gitlink(ce);\n-\t\treturn;\n-\t}\n+\t} else if (match_pathspec(&pathspec, name.buf, name.len,\n+\t\t\t\t len, ps_matched,\n+\t\t\t\t S_ISDIR(ce->ce_mode) ||\n+\t\t\t\t S_ISGITLINK(ce->ce_mode))) {\n+\t\tif (tag && *tag && show_valid_bit &&\n+\t\t (ce->ce_flags & CE_VALID)) {\n+\t\t\tstatic char alttag[4];\n+\t\t\tmemcpy(alttag, tag, 3);\n+\t\t\tif (isalpha(tag[0]))\n+\t\t\t\talttag[0] = tolower(tag[0]);\n+\t\t\telse if (tag[0] == '?')\n+\t\t\t\talttag[0] = '!';\n+\t\t\telse {\n+\t\t\t\talttag[0] = 'v';\n+\t\t\t\talttag[1] = tag[0];\n+\t\t\t\talttag[2] = ' ';\n+\t\t\t\talttag[3] = 0;\n+\t\t\t}\n+\t\t\ttag = alttag;\n+\t\t}\n \n-\tif (tag && *tag && show_valid_bit &&\n-\t (ce->ce_flags & CE_VALID)) {\n-\t\tstatic char alttag[4];\n-\t\tmemcpy(alttag, tag, 3);\n-\t\tif (isalpha(tag[0]))\n-\t\t\talttag[0] = tolower(tag[0]);\n-\t\telse if (tag[0] == '?')\n-\t\t\talttag[0] = '!';\n-\t\telse {\n-\t\t\talttag[0] = 'v';\n-\t\t\talttag[1] = tag[0];\n-\t\t\talttag[2] = ' ';\n-\t\t\talttag[3] = 0;\n+\t\tif (!show_stage) {\n+\t\t\tfputs(tag, stdout);\n+\t\t} else {\n+\t\t\tprintf(\"%s%06o %s %d\\t\",\n+\t\t\t tag,\n+\t\t\t ce->ce_mode,\n+\t\t\t find_unique_abbrev(ce->sha1,abbrev),\n+\t\t\t ce_stage(ce));\n+\t\t}\n+\t\twrite_eolinfo(ce, ce->name);\n+\t\twrite_name(ce->name);\n+\t\tif (debug_mode) {\n+\t\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n+\n+\t\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n+\t\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n+\t\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n+\t\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n+\t\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n \t\t}\n-\t\ttag = alttag;\n \t}\n \n-\tif (!show_stage) {\n-\t\tfputs(tag, stdout);\n-\t} else {\n-\t\tprintf(\"%s%06o %s %d\\t\",\n-\t\t tag,\n-\t\t ce->ce_mode,\n-\t\t find_unique_abbrev(ce->sha1,abbrev),\n-\t\t ce_stage(ce));\n-\t}\n-\twrite_eolinfo(ce, ce->name);\n-\twrite_name(ce->name);\n-\tif (debug_mode) {\n-\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n-\n-\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n-\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n-\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n-\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n-\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n-\t}\n+\tstrbuf_release(&name);\n }\n \n static void show_ru_info(void)\n@@ -568,27 +597,28 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\tsubmodule_prefix = getenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT);\n \n \tif (recurse_submodules &&\n-\t (show_stage || show_deleted || show_others || show_unmerged ||\n-\t show_killed || show_modified || show_resolve_undo ||\n-\t show_valid_bit || show_tag || show_eol))\n-\t\tdie(\"ls-files --recurse-submodules can only be used in \"\n-\t\t \"--cached mode\");\n+\t (show_deleted || show_others || show_unmerged ||\n+\t show_killed || show_modified || show_resolve_undo))\n+\t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n \n \tif (recurse_submodules && error_unmatch)\n \t\tdie(\"ls-files --recurse-submodules does not support \"\n \t\t \"--error-unmatch\");\n \n-\tif (recurse_submodules && argc)\n-\t\tdie(\"ls-files --recurse-submodules does not support path \"\n-\t\t \"arguments\");\n-\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\n \t\t prefix, argv);\n \n-\t/* Find common prefix for all pathspec's */\n-\tmax_prefix = common_prefix(&pathspec);\n+\t/*\n+\t * Find common prefix for all pathspec's\n+\t * This is used as a performance optimization which unfortunately cannot\n+\t * be done when recursing into submodules\n+\t */\n+\tif (recurse_submodules)\n+\t\tmax_prefix = NULL;\n+\telse\n+\t\tmax_prefix = common_prefix(&pathspec);\n \tmax_prefix_len = max_prefix ? strlen(max_prefix) : 0;\n \n \t/* Treat unmatching pathspec elements as errors */\ndiff --git a/dir.c b/dir.c\nindex 0ea235f..28e9736 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -207,8 +207,9 @@ int within_depth(const char *name, int namelen,\n \treturn 1;\n }\n \n-#define DO_MATCH_EXCLUDE 1\n-#define DO_MATCH_DIRECTORY 2\n+#define DO_MATCH_EXCLUDE (1<<0)\n+#define DO_MATCH_DIRECTORY (1<<1)\n+#define DO_MATCH_SUBMODULE (1<<2)\n \n /*\n * Does 'match' match the given name?\n@@ -283,6 +284,32 @@ static int match_pathspec_item(const struct pathspec_item *item, int prefix,\n \t\t\t item->nowildcard_len - prefix))\n \t\treturn MATCHED_FNMATCH;\n \n+\t/* Perform checks to see if \"name\" is a super set of the pathspec */\n+\tif (flags & DO_MATCH_SUBMODULE) {\n+\t\t/* name is a literal prefix of the pathspec */\n+\t\tif ((namelen < matchlen) &&\n+\t\t (match[namelen] == '/') &&\n+\t\t !ps_strncmp(item, match, name, namelen))\n+\t\t\treturn MATCHED_RECURSIVELY;\n+\n+\t\t/* name\" doesn't match up to the first wild character */\n+\t\tif (item->nowildcard_len < item->len &&\n+\t\t ps_strncmp(item, match, name,\n+\t\t\t item->nowildcard_len - prefix))\n+\t\t\treturn 0;\n+\n+\t\t/*\n+\t\t * Here is where we would perform a wildmatch to check if\n+\t\t * \"name\" can be matched as a directory (or a prefix) against\n+\t\t * the pathspec. Since wildmatch doesn't have this capability\n+\t\t * at the present we have to punt and say that it is a match,\n+\t\t * potentially returning a false positive\n+\t\t * The submodules themselves will be able to perform more\n+\t\t * accurate matching to determine if the pathspec matches.\n+\t\t */\n+\t\treturn MATCHED_RECURSIVELY;\n+\t}\n+\n \treturn 0;\n }\n \n@@ -386,6 +413,21 @@ int match_pathspec(const struct pathspec *ps,\n \treturn negative ? 0 : positive;\n }\n \n+/**\n+ * Check if a submodule is a superset of the pathspec\n+ */\n+int submodule_path_match(const struct pathspec *ps,\n+\t\t\t const char *submodule_name,\n+\t\t\t char *seen)\n+{\n+\tint matched = do_match_pathspec(ps, submodule_name,\n+\t\t\t\t\tstrlen(submodule_name),\n+\t\t\t\t\t0, seen,\n+\t\t\t\t\tDO_MATCH_DIRECTORY |\n+\t\t\t\t\tDO_MATCH_SUBMODULE);\n+\treturn matched;\n+}\n+\n int report_path_error(const char *ps_matched,\n \t\t const struct pathspec *pathspec,\n \t\t const char *prefix)\ndiff --git a/dir.h b/dir.h\nindex da1a858..97c83bb 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item,\n \t\t const char *pattern, const char *string,\n \t\t int prefix);\n \n+extern int submodule_path_match(const struct pathspec *ps,\n+\t\t\t\tconst char *submodule_name,\n+\t\t\t\tchar *seen);\n+\n static inline int ce_path_match(const struct cache_entry *ce,\n \t\t\t\tconst struct pathspec *pathspec,\n \t\t\t\tchar *seen)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex caf3815..4a51d38 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'ls-files correctly outputs files in submodule with -z' '\n+\tcat | tr \"\\n\" \"\\0\" >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules -z >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'ls-files does not output files not added to a repo' '\n \tcat >expect <<-\\EOF &&\n \t.gitmodules\n@@ -69,9 +81,111 @@ test_expect_success 'ls-files recurses more than 1 level' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success '--recurse-submodules does not support using path arguments' '\n-\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n-\ttest_i18ngrep \"does not support path arguments\" actual\n+test_expect_success '--recurse-submodules and pathspecs setup' '\n+\techo e >submodule/subsub/e.txt &&\n+\tgit -C submodule/subsub add e.txt &&\n+\tgit -C submodule/subsub commit -m \"adding e.txt\" &&\n+\techo f >submodule/f.TXT &&\n+\techo g >submodule/g.txt &&\n+\tgit -C submodule add f.TXT g.txt &&\n+\tgit -C submodule commit -m \"add f and g\" &&\n+\techo h >h.txt &&\n+\tmkdir sib &&\n+\techo sib >sib/file &&\n+\tgit add h.txt sib/file &&\n+\tgit commit -m \"add h and sib/file\" &&\n+\tgit init sub &&\n+\techo sub >sub/file &&\n+\tgit -C sub add file &&\n+\tgit -C sub commit -m \"add file\" &&\n+\tgit submodule add ./sub &&\n+\tgit commit -m \"added sub\" &&\n+\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\th.txt\n+\tsib/file\n+\tsub/file\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/d\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual &&\n+\tcat actual &&\n+\tgit ls-files --recurse-submodules \"*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" \":(exclude)submodule/subsub/*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"sub\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su*/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su?/file\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsib/file\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"s??/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s???file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s*file\" >actual &&\n+\ttest_cmp expect actual\n '\n \n test_expect_success '--recurse-submodules does not support --error-unmatch' '\n@@ -82,18 +196,14 @@ test_expect_success '--recurse-submodules does not support --error-unmatch' '\n test_incompatible_with_recurse_submodules () {\n \ttest_expect_success \"--recurse-submodules and $1 are incompatible\" \"\n \t\ttest_must_fail git ls-files --recurse-submodules $1 2>actual &&\n-\t\ttest_i18ngrep 'can only be used in --cached mode' actual\n+\t\ttest_i18ngrep 'unsupported mode' actual\n \t\"\n }\n \n-test_incompatible_with_recurse_submodules -v\n-test_incompatible_with_recurse_submodules -t\n test_incompatible_with_recurse_submodules --deleted\n test_incompatible_with_recurse_submodules --modified\n test_incompatible_with_recurse_submodules --others\n-test_incompatible_with_recurse_submodules --stage\n test_incompatible_with_recurse_submodules --killed\n test_incompatible_with_recurse_submodules --unmerged\n-test_incompatible_with_recurse_submodules --eol\n \n test_done\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302505","messageId":"20160925071744.2qcz7cntc7xxpjqv@sigill.intra.peff.net","threadId":"44147","inReplyTo":"1474676014-134568-1-git-send-email-bmwill@google.com","subject":"Re: [PATCH 0/3] recursive support for ls-files","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-09-25T07:17:44Z","receivedAt":"2016-09-25T07:17:53Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Sep 23, 2016 at 05:13:31PM -0700, Brandon Williams wrote:\n\n> After looking at the feedback I rerolled a few things, in particular the\n> --submodule_prefix option that existed to give a submodule context about where\n> it had been invoked from. People didn't seem to like the idea of exposing this\n> to the users (yet anyways) so I removed it as an option and instead have it\n> being passed to a child process via an environment variable\n> GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to\n> external users at the moment.\n\nI think we can still have it as a command-line argument and declare it\ninternal. It's not like environment variables cannot also be set by our\ncallers. :)\n\nI don't mind it as an environment variable, though. In some ways it\nmakes things easier. I just think \"internal versus external\" and the\nexact implementation are orthogonal.\n\n> Also fixed a bug (and added a test) for the -z options as pointed out by Jeff\n> King.\n\nHmm. It is broken after patch 2, and then fixed in patch 3. Usually we'd\ntry not to have a broken state in the history. It's less important in\nthis case, because the breakage is not a regression\n(--recurse-submodules is a new feature, so you could consider it \"not\nworking\" until the 3rd patch). But I think it's still a good rule to\nfollow, because it makes the commits easier to review, look at later,\netc.\n\nFor that matter, I do not understand why options like \"-s\" get enabled\nin patch 3. I do not mind them starting as disabled in patch 2, but it\nseems like \"pass along some known-safe options\" should be its own patch\nsomewhere between patches 2 and 3.\n\nThere are some other options that are ignored (neither disabled nor\npassed along to children). Most of them are related to exclusions, which\nI _think_ are safe to ignore (they do not do anything interesting unless\nyou specify \"-o\", which is explicitly disabled). I'm not sure about\n--with-tree, though (or what it would even mean in the context of\nrecursing).\n\n-Peff\n"},{"id":"302524","messageId":"20160925163244.GB66080@google.com","threadId":"44147","inReplyTo":"20160925071744.2qcz7cntc7xxpjqv@sigill.intra.peff.net","subject":"Re: [PATCH 0/3] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-25T16:32:44Z","receivedAt":"2016-09-25T16:32:53Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/25, Jeff King wrote:\n> On Fri, Sep 23, 2016 at 05:13:31PM -0700, Brandon Williams wrote:\n> \n> > After looking at the feedback I rerolled a few things, in particular the\n> > --submodule_prefix option that existed to give a submodule context about where\n> > it had been invoked from. People didn't seem to like the idea of exposing this\n> > to the users (yet anyways) so I removed it as an option and instead have it\n> > being passed to a child process via an environment variable\n> > GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to\n> > external users at the moment.\n> \n> I think we can still have it as a command-line argument and declare it\n> internal. It's not like environment variables cannot also be set by our\n> callers. :)\n> \n> I don't mind it as an environment variable, though. In some ways it\n> makes things easier. I just think \"internal versus external\" and the\n> exact implementation are orthogonal.\n> \n\nWe may still want it to be an option at some point in the future. This\nway we can revisit making it an option once we know more about the other\nuses it could have (aside from just being for submodules as someone\nsuggested).\n\n> > Also fixed a bug (and added a test) for the -z options as pointed out by Jeff\n> > King.\n> \n> Hmm. It is broken after patch 2, and then fixed in patch 3. Usually we'd\n> try not to have a broken state in the history. It's less important in\n> this case, because the breakage is not a regression\n> (--recurse-submodules is a new feature, so you could consider it \"not\n> working\" until the 3rd patch). But I think it's still a good rule to\n> follow, because it makes the commits easier to review, look at later,\n> etc.\n> \n> For that matter, I do not understand why options like \"-s\" get enabled\n> in patch 3. I do not mind them starting as disabled in patch 2, but it\n> seems like \"pass along some known-safe options\" should be its own patch\n> somewhere between patches 2 and 3.\n\nI'll keep that in mind for future patches. I figured that since it was\nfixed in the end that would be fine but if things shouldn't be broken at\nany state in the patch series I'll make sure to not do that in the\nfuture.\n\n> There are some other options that are ignored (neither disabled nor\n> passed along to children). Most of them are related to exclusions, which\n> I _think_ are safe to ignore (they do not do anything interesting unless\n> you specify \"-o\", which is explicitly disabled). I'm not sure about\n> --with-tree, though (or what it would even mean in the context of\n> recursing).\n\nThese other features that are disabled now could be enabled in a future\npatch. You're right though I'd have to think about the --with-tree\noption a bit more and what it would mean with submodules.\n\n-Brandon\n"},{"id":"302530","messageId":"xmqqzimvygdt.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"20160925163244.GB66080@google.com","subject":"Re: [PATCH 0/3] recursive support for ls-files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-25T18:38:06Z","receivedAt":"2016-09-25T18:38:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> On 09/25, Jeff King wrote:\n>> On Fri, Sep 23, 2016 at 05:13:31PM -0700, Brandon Williams wrote:\n>> \n>> > After looking at the feedback I rerolled a few things, in particular the\n>> > --submodule_prefix option that existed to give a submodule context about where\n>> > it had been invoked from. People didn't seem to like the idea of exposing this\n>> > to the users (yet anyways) so I removed it as an option and instead have it\n>> > being passed to a child process via an environment variable\n>> > GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to\n>> > external users at the moment.\n>> \n>> I think we can still have it as a command-line argument and declare it\n>> internal. It's not like environment variables cannot also be set by our\n>> callers. :)\n>> \n>> I don't mind it as an environment variable, though. In some ways it\n>> makes things easier. I just think \"internal versus external\" and the\n>> exact implementation are orthogonal.\n>\n> We may still want it to be an option at some point in the future. This\n> way we can revisit making it an option once we know more about the other\n> uses it could have (aside from just being for submodules as someone\n> suggested).\n\nI do not think it makes too much of a difference between environment\nand command line option. We need an update to the \"git\" potty to\nsay \"you told me to use the submodule-prefix feature, but this\nsubcommand is not prepared to accept it (yet)\" and cause it to error\nout either way, which would mean that a series that introduces the\nfeature needs to touch \"git.c\" anyway, so I would have expected us\nto add command line option first, simply because \"git.c\" is where it\nhappens, optionally with the support for the environment variable,\nnot the other way around.\n\n>> > Also fixed a bug (and added a test) for the -z options as pointed out by Jeff\n>> > King.\n>> \n>> Hmm. It is broken after patch 2, and then fixed in patch 3. Usually we'd\n>> try not to have a broken state in the history. It's less important in\n>> this case, because the breakage is not a regression\n>> (--recurse-submodules is a new feature, so you could consider it \"not\n>> working\" until the 3rd patch). But I think it's still a good rule to\n>> follow, because it makes the commits easier to review, look at later,\n>> etc.\n>> \n>> For that matter, I do not understand why options like \"-s\" get enabled\n>> in patch 3. I do not mind them starting as disabled in patch 2, but it\n>> seems like \"pass along some known-safe options\" should be its own patch\n>> somewhere between patches 2 and 3.\n\nYes, exactly.\n\nAn obvious lazy way out to avoid breakage-in-the-middle and make\nincremental progress would be to squash everything into one patch,\nbut we should and we should be able to do better.\n\nI'd imagine this three-patch series would be more pleasant for\nfuture readers if it were structured like:\n\n [1/3] introduces the submodule-prefix as a global feature; at the\n least it needs a way to invoke (either an environment, or an\n option to \"git\" potty, or both) and prevent mistakes by\n erroring out when it is attempted to call a subcommand that\n does not support the feature (yet).\n\n [2/3] adds the --recurse-submodule feature in a limited form to\n \"ls-files\". I'd suggest for this step to pass through all\n options and arguments that are safe and reasonably useful\n to pass through without needing anything more than \"ah, this\n option was given, so let's stuff it to the argv-array\". An\n attempt to give things that are not yet passed through until\n 3/3 to lead to an error that says it is not allowed (yet).\n\n [3-N] each of the remaining steps after 3/N adds support for one\n more thing to be passed that 2/3 refrained from doing, by\n doing more than just \"pass it in argv-array\", and then remove\n the \"not yet supported\" error that added by 2/3 for that one\n thing. The first of these \"more things\" would be to support\n pathspecs as the receiving side would need code changes for\n the matching logic. There may be more, or there may be\n nothing else that requires 4/N, 5/N, etc.\n\n>> There are some other options that are ignored (neither disabled nor\n>> passed along to children). Most of them are related to exclusions, which\n>> I _think_ are safe to ignore (they do not do anything interesting unless\n>> you specify \"-o\", which is explicitly disabled).\n\nSure. I agree that some options fall outside of \"safe and reasonably\nuseful to pass through\" criteria and it is OK not to support passing\nthem through. I however think we should detect mistakes in the\ncaller, though.\n\nI wonder if this will be common enough in the future that we are\nbetter off adding a bit or two to the parse-options infrastructure.\nThinking out loud, would an enhancement like this be sufficient for\nthis particular series, and still useful in more general cases?\n\n - allow the caller of parse_options() to mark each entry with a\n handful of bits with no meaning to parse-options API. \"ls-files\"\n may use this mechanism to mark options that to be passed through\n and options that must be rejected when --recurse-submodules is\n asked.\n\n - parse_options() will mark each entry as \"this option was given\"\n while doing its work. Note that \"--no-foo\" counts as \"option\n 'foo' was given\".\n\nWith these, after your call to parse_options() returns, you could\nnotice that recurse-submodules was asked for by inspecting your own\n\"static int recurse_submodules\" global variable, and then iterate\nover builtin_ls_files_options[] array yourself to see if an option\nthat you marked with \"must be rejected while recursing\" bit using\nmechanism #1 was seen by parse_options() by relying on mechanism #2.\n\nYou could also add a new helper macro to parse-options API that\niterates over the options[] array, i.e.\n\n\tfor_each_parse_options_array(opt) {\n\t\tif (!opt_was_used(opt))\n \tcontinue;\n\t\tif (opt_custom_bit(opt) & NOT_IN_RECURSIVE)\n \tdie(\"'%s' cannot be used with --recurse-submodules\",\n\t\t\t parse_options_name(opt))\n\t}\n\nor something like that, but we'd need to gain experience with the\nmechanism #1 and #2 first before deciding if such a helper is useful\n(iow, I do not think we need it from day one, if we go this route).\n\n"},{"id":"302546","messageId":"xmqqintjwo2x.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1474676014-134568-2-git-send-email-bmwill@google.com","subject":"Re: [PATCH 1/3 v3] submodules: make submodule-prefix option an envvar","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-25T23:34:46Z","receivedAt":"2016-09-25T23:34:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> Add a submodule-prefix enviorment variable\n\nenvironment?\n\n> 'GIT_INTERNAL_SUBMODULE_PREFIX' which can be used by commands which have\n> --recurse-submodule options.\n>\n> Signed-off-by: Brandon Williams \n> ---\n> cache.h | 1 +\n> environment.c | 1 +\n> 2 files changed, 2 insertions(+)\n>\n> diff --git a/cache.h b/cache.h\n> index 3556326..ae88a35 100644\n> --- a/cache.h\n> +++ b/cache.h\n> @@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)\n> #define GIT_NAMESPACE_ENVIRONMENT \"GIT_NAMESPACE\"\n> #define GIT_WORK_TREE_ENVIRONMENT \"GIT_WORK_TREE\"\n> #define GIT_PREFIX_ENVIRONMENT \"GIT_PREFIX\"\n> +#define GIT_SUBMODULE_PREFIX_ENVIRONMENT \"GIT_INTERNAL_SUBMODULE_PREFIX\"\n> #define DEFAULT_GIT_DIR_ENVIRONMENT \".git\"\n> #define DB_ENVIRONMENT \"GIT_OBJECT_DIRECTORY\"\n> #define INDEX_ENVIRONMENT \"GIT_INDEX_FILE\"\n> diff --git a/environment.c b/environment.c\n> index ca72464..7380815 100644\n> --- a/environment.c\n> +++ b/environment.c\n> @@ -120,6 +120,7 @@ const char * const local_repo_env[] = {\n> \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n> \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n> \tGIT_PREFIX_ENVIRONMENT,\n> +\tGIT_SUBMODULE_PREFIX_ENVIRONMENT,\n> \tGIT_SHALLOW_FILE_ENVIRONMENT,\n> \tGIT_COMMON_DIR_ENVIRONMENT,\n> \tNULL\n"},{"id":"302586","messageId":"20160926170429.GA3624@google.com","threadId":"44147","inReplyTo":"xmqqzimvygdt.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 0/3] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-26T17:04:29Z","receivedAt":"2016-09-26T17:04:40Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/25, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > On 09/25, Jeff King wrote:\n> >> On Fri, Sep 23, 2016 at 05:13:31PM -0700, Brandon Williams wrote:\n> >> \n> >> > After looking at the feedback I rerolled a few things, in particular the\n> >> > --submodule_prefix option that existed to give a submodule context about where\n> >> > it had been invoked from. People didn't seem to like the idea of exposing this\n> >> > to the users (yet anyways) so I removed it as an option and instead have it\n> >> > being passed to a child process via an environment variable\n> >> > GIT_INTERNAL_SUBMODULE_PREFIX. This way we don't have to support anything to\n> >> > external users at the moment.\n> >> \n> >> I think we can still have it as a command-line argument and declare it\n> >> internal. It's not like environment variables cannot also be set by our\n> >> callers. :)\n> >> \n> >> I don't mind it as an environment variable, though. In some ways it\n> >> makes things easier. I just think \"internal versus external\" and the\n> >> exact implementation are orthogonal.\n> >\n> > We may still want it to be an option at some point in the future. This\n> > way we can revisit making it an option once we know more about the other\n> > uses it could have (aside from just being for submodules as someone\n> > suggested).\n> \n> I do not think it makes too much of a difference between environment\n> and command line option. We need an update to the \"git\" potty to\n> say \"you told me to use the submodule-prefix feature, but this\n> subcommand is not prepared to accept it (yet)\" and cause it to error\n> out either way, which would mean that a series that introduces the\n> feature needs to touch \"git.c\" anyway, so I would have expected us\n> to add command line option first, simply because \"git.c\" is where it\n> happens, optionally with the support for the environment variable,\n> not the other way around.\n\nIn a previous email you mentioned that this feature should be completely\nhidden from users, which is why I removed the command line option for\nthis latest series. If that isn't what you intended that I can\ndefinitely add the option to git.c. And you would rather we perform the\nchecking in git.c to see if a subcommand supports the prefix versus\nsilently ignoring it if it hasn't? I'm assuming this checking would\nalso be done in git.c?\n\n> \n> >> > Also fixed a bug (and added a test) for the -z options as pointed out by Jeff\n> >> > King.\n> >> \n> >> Hmm. It is broken after patch 2, and then fixed in patch 3. Usually we'd\n> >> try not to have a broken state in the history. It's less important in\n> >> this case, because the breakage is not a regression\n> >> (--recurse-submodules is a new feature, so you could consider it \"not\n> >> working\" until the 3rd patch). But I think it's still a good rule to\n> >> follow, because it makes the commits easier to review, look at later,\n> >> etc.\n> >> \n> >> For that matter, I do not understand why options like \"-s\" get enabled\n> >> in patch 3. I do not mind them starting as disabled in patch 2, but it\n> >> seems like \"pass along some known-safe options\" should be its own patch\n> >> somewhere between patches 2 and 3.\n> \n> Yes, exactly.\n> \n> An obvious lazy way out to avoid breakage-in-the-middle and make\n> incremental progress would be to squash everything into one patch,\n> but we should and we should be able to do better.\n> \n> I'd imagine this three-patch series would be more pleasant for\n> future readers if it were structured like:\n> \n> [1/3] introduces the submodule-prefix as a global feature; at the\n> least it needs a way to invoke (either an environment, or an\n> option to \"git\" potty, or both) and prevent mistakes by\n> erroring out when it is attempted to call a subcommand that\n> does not support the feature (yet).\n> \n> [2/3] adds the --recurse-submodule feature in a limited form to\n> \"ls-files\". I'd suggest for this step to pass through all\n> options and arguments that are safe and reasonably useful\n> to pass through without needing anything more than \"ah, this\n> option was given, so let's stuff it to the argv-array\". An\n> attempt to give things that are not yet passed through until\n> 3/3 to lead to an error that says it is not allowed (yet).\n> \n> [3-N] each of the remaining steps after 3/N adds support for one\n> more thing to be passed that 2/3 refrained from doing, by\n> doing more than just \"pass it in argv-array\", and then remove\n> the \"not yet supported\" error that added by 2/3 for that one\n> thing. The first of these \"more things\" would be to support\n> pathspecs as the receiving side would need code changes for\n> the matching logic. There may be more, or there may be\n> nothing else that requires 4/N, 5/N, etc.\n\nI can do another rework and structure the patch series more inline with\nwhat you are suggesting here.\n\n-- \nBrandon Williams\n"},{"id":"302609","messageId":"xmqqponqseya.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"20160926170429.GA3624@google.com","subject":"Re: [PATCH 0/3] recursive support for ls-files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-26T18:17:49Z","receivedAt":"2016-09-26T18:17:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> In a previous email you mentioned that this feature should be completely\n> hidden from users, which is why I removed the command line option for\n> this latest series.\n\nI may have said something like that; I do not recall, though, so a\nmore accurate description might be \"I may have said something that\ncan be (mis)interpreted like so\". Sorry for the confusion.\n\nIn any case, an environment is not \"completely hidden from users\",\nso it is not fundamentallly different from a command line option\nanyway ;-)\n\n> If that isn't what you intended that I can\n> definitely add the option to git.c. And you would rather we perform the\n> checking in git.c to see if a subcommand supports the prefix versus\n> silently ignoring it if it hasn't? I'm assuming this checking would\n> also be done in git.c?\n\nI actually do not care strongly _where_ the check happens. It was\njust that in the subcommand dispatcher would be the single place\nthat is easiest-to-implement to perform that check, that made me\nsuggest that. We already have various bits like NEEDS_WORK_TREE,\nRUN_SETUP, etc. so REJECT_EXTERNAL_PREFIX (or whatever its name be;\nI do not offhand recall current proposal) bit would fit there\nnaturally, I would think. Of course, non-built-in commands need to\nprotect themselves separately, if they want to.\n"},{"id":"302611","messageId":"20160926183811.GA5123@google.com","threadId":"44147","inReplyTo":"xmqqponqseya.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 0/3] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-26T18:38:11Z","receivedAt":"2016-09-26T18:38:35Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/26, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > In a previous email you mentioned that this feature should be completely\n> > hidden from users, which is why I removed the command line option for\n> > this latest series.\n> \n> I may have said something like that; I do not recall, though, so a\n> more accurate description might be \"I may have said something that\n> can be (mis)interpreted like so\". Sorry for the confusion.\n> \n> In any case, an environment is not \"completely hidden from users\",\n> so it is not fundamentallly different from a command line option\n> anyway ;-)\n\nNo worries, I'm still trying to get a hang of all this :)\nTrue, that wouldn't be completely hidden either.\n\n> > If that isn't what you intended that I can\n> > definitely add the option to git.c. And you would rather we perform the\n> > checking in git.c to see if a subcommand supports the prefix versus\n> > silently ignoring it if it hasn't? I'm assuming this checking would\n> > also be done in git.c?\n> \n> I actually do not care strongly _where_ the check happens. It was\n> just that in the subcommand dispatcher would be the single place\n> that is easiest-to-implement to perform that check, that made me\n> suggest that. We already have various bits like NEEDS_WORK_TREE,\n> RUN_SETUP, etc. so REJECT_EXTERNAL_PREFIX (or whatever its name be;\n> I do not offhand recall current proposal) bit would fit there\n> naturally, I would think. Of course, non-built-in commands need to\n> protect themselves separately, if they want to.\n\nThat makes sense. I have an idea of where the check could be made. And\nwith those flags it may make sense to have the flag be an indicator that\nthe builtin is ready for submodule type commands ie SUPPORTS_SUBMODULES\nor something along those lines. That way we only need to add the flag\nto each command as we go (instead of all commands which don't support\nit) and just make sure that the flag is set if the submodule prefix\noption is being used. How does that sound?\n\n-- \nBrandon Williams\n"},{"id":"302612","messageId":"xmqqh992sdjq.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"20160926183811.GA5123@google.com","subject":"Re: [PATCH 0/3] recursive support for ls-files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-26T18:48:09Z","receivedAt":"2016-09-26T18:48:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> or something along those lines. That way we only need to add the flag\n> to each command as we go ...\n\nSounds good. Thanks.\n"},{"id":"302655","messageId":"1474930003-83750-1-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474676014-134568-1-git-send-email-bmwill@google.com","subject":"[PATCH 0/4 v4] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-26T22:46:39Z","receivedAt":"2016-09-26T22:47:09Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"A couple things have changed in v4:\n\n- Restructured the patch series to prevent a breakage mid-way.\n- Added an additional patch in the middle to pass through safe options. This\n way the series is structured in a more coherent manor.\n- Added --submodule-prefix to top-level git.c\n\nHopefully this series addresses some of issues brought up in v3\n\nBrandon Williams (4):\n submodules: make submodule-prefix option\n ls-files: optionally recurse into submodules\n ls-files: pass through safe options for --recurse-submodules\n ls-files: add pathspec matching for submodules\n\n Documentation/git-ls-files.txt | 7 +-\n Documentation/git.txt | 5 +\n builtin/ls-files.c | 187 ++++++++++++++++++++++-------\n cache.h | 1 +\n dir.c | 46 +++++++-\n dir.h | 4 +\n environment.c | 1 +\n git.c | 21 +++-\n t/t3007-ls-files-recurse-submodules.sh | 209 +++++++++++++++++++++++++++++++++\n 9 files changed, 437 insertions(+), 44 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302656","messageId":"1474930003-83750-2-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474930003-83750-1-git-send-email-bmwill@google.com","subject":"[PATCH 1/4 v4] submodules: make submodule-prefix option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-26T22:46:40Z","receivedAt":"2016-09-26T22:47:14Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add a submodule-prefix environment variable\n'GIT_INTERNAL_SUBMODULE_PREFIX' which can be used by commands which have\n--recurse-submodule options to give context to submodules about how they\nwere invoked. This option is only allowed for builtins which have\nsubmodule support.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git.txt | 5 +++++\n cache.h | 1 +\n environment.c | 1 +\n git.c | 19 +++++++++++++++++++\n 4 files changed, 26 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 7913fc2..d29967a 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -13,6 +13,7 @@ SYNOPSIS\n [--exec-path[=]] [--html-path] [--man-path] [--info-path]\n [-p|--paginate|--no-pager] [--no-replace-objects] [--bare]\n [--git-dir=] [--work-tree=] [--namespace=]\n+ [--submodule-prefix=]\n []\n \n DESCRIPTION\n@@ -601,6 +602,10 @@ foo.bar= ...`) sets `foo.bar` to the empty string.\n \tdetails. Equivalent to setting the `GIT_NAMESPACE` environment\n \tvariable.\n \n+--submodule-prefix=::\n+\tSet a prefix which gives submodules context about the superproject that\n+\tinvoked it. Only allowed for commands which support submodules.\n+\n --bare::\n \tTreat the repository as a bare repository. If GIT_DIR\n \tenvironment is not set, it is set to the current working\ndiff --git a/cache.h b/cache.h\nindex 3556326..ae88a35 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)\n #define GIT_NAMESPACE_ENVIRONMENT \"GIT_NAMESPACE\"\n #define GIT_WORK_TREE_ENVIRONMENT \"GIT_WORK_TREE\"\n #define GIT_PREFIX_ENVIRONMENT \"GIT_PREFIX\"\n+#define GIT_SUBMODULE_PREFIX_ENVIRONMENT \"GIT_INTERNAL_SUBMODULE_PREFIX\"\n #define DEFAULT_GIT_DIR_ENVIRONMENT \".git\"\n #define DB_ENVIRONMENT \"GIT_OBJECT_DIRECTORY\"\n #define INDEX_ENVIRONMENT \"GIT_INDEX_FILE\"\ndiff --git a/environment.c b/environment.c\nindex ca72464..7380815 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -120,6 +120,7 @@ const char * const local_repo_env[] = {\n \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n \tGIT_PREFIX_ENVIRONMENT,\n+\tGIT_SUBMODULE_PREFIX_ENVIRONMENT,\n \tGIT_SHALLOW_FILE_ENVIRONMENT,\n \tGIT_COMMON_DIR_ENVIRONMENT,\n \tNULL\ndiff --git a/git.c b/git.c\nindex 1c61151..b2b096a 100644\n--- a/git.c\n+++ b/git.c\n@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n \t\t\tsetenv(GIT_WORK_TREE_ENVIRONMENT, cmd, 1);\n \t\t\tif (envchanged)\n \t\t\t\t*envchanged = 1;\n+\t\t} else if (!strcmp(cmd, \"--submodule-prefix\")) {\n+\t\t\tif (*argc < 2) {\n+\t\t\t\tfprintf(stderr, \"No prefix given for --submodule-prefix.\\n\" );\n+\t\t\t\tusage(git_usage_string);\n+\t\t\t}\n+\t\t\tsetenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT, (*argv)[1], 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n+\t\t\t(*argv)++;\n+\t\t\t(*argc)--;\n+\t\t} else if (skip_prefix(cmd, \"--submodule-prefix=\", &cmd)) {\n+\t\t\tsetenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT, cmd, 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--bare\")) {\n \t\t\tchar *cwd = xgetcwd();\n \t\t\tis_bare_repository_cfg = 1;\n@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)\n * RUN_SETUP for reading from the configuration file.\n */\n #define NEED_WORK_TREE\t\t(1<<3)\n+#define SUPPORT_SUBMODULES\t(1<<4)\n \n struct cmd_struct {\n \tconst char *cmd;\n@@ -344,6 +359,10 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n \t}\n \tcommit_pager_choice();\n \n+\tif (!help && (getenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT) &&\n+\t\t !(p->option & SUPPORT_SUBMODULES)))\n+\t\tdie(\"%s doesn't support submodules\", p->cmd);\n+\n \tif (!help && p->option & NEED_WORK_TREE)\n \t\tsetup_work_tree();\n \n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302657","messageId":"1474930003-83750-3-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474930003-83750-1-git-send-email-bmwill@google.com","subject":"[PATCH 2/4 v4] ls-files: optionally recurse into submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-26T22:46:41Z","receivedAt":"2016-09-26T22:47:20Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Allow ls-files to recognize submodules in order to retrieve a list of\nfiles from a repository's submodules. This is done by forking off a\nprocess to recursively call ls-files on all submodules. Use top-level\n--submodule_prefix option to pass a path to the submodule which it can\nuse to prepend to output or pathspec matching logic.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git-ls-files.txt | 7 +-\n builtin/ls-files.c | 143 ++++++++++++++++++++++++---------\n git.c | 2 +-\n t/t3007-ls-files-recurse-submodules.sh | 100 +++++++++++++++++++++++\n 4 files changed, 212 insertions(+), 40 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\ndiff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\nindex 0d933ac..446209e 100644\n--- a/Documentation/git-ls-files.txt\n+++ b/Documentation/git-ls-files.txt\n@@ -18,7 +18,8 @@ SYNOPSIS\n \t\t[--exclude-per-directory=]\n \t\t[--exclude-standard]\n \t\t[--error-unmatch] [--with-tree=]\n-\t\t[--full-name] [--abbrev] [--] [...]\n+\t\t[--full-name] [--recurse-submodules]\n+\t\t[--abbrev] [--] [...]\n \n DESCRIPTION\n -----------\n@@ -137,6 +138,10 @@ a space) at the start of each line:\n \toption forces paths to be output relative to the project\n \ttop directory.\n \n+--recurse-submodules::\n+\tRecursively calls ls-files on each submodule in the repository.\n+\tCurrently there is only support for the --cached mode.\n+\n --abbrev[=]::\n \tInstead of showing the full 40-byte hexadecimal object\n \tlines, show only a partial prefix.\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 00ea91a..d4bfc60 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -14,6 +14,7 @@\n #include \"resolve-undo.h\"\n #include \"string-list.h\"\n #include \"pathspec.h\"\n+#include \"run-command.h\"\n \n static int abbrev;\n static int show_deleted;\n@@ -28,6 +29,8 @@ static int show_valid_bit;\n static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n+static int recurse_submodules;\n+static const char *submodule_prefix;\n \n static const char *prefix;\n static int max_prefix_len;\n@@ -68,6 +71,21 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)\n static void write_name(const char *name)\n {\n \t/*\n+\t * NEEDSWORK: To make this thread-safe, full_name would have to be owned\n+\t * by the caller.\n+\t *\n+\t * full_name get reused across output lines to minimize the allocation\n+\t * churn.\n+\t */\n+\tstatic struct strbuf full_name = STRBUF_INIT;\n+\tif (submodule_prefix && *submodule_prefix) {\n+\t\tstrbuf_reset(&full_name);\n+\t\tstrbuf_addstr(&full_name, submodule_prefix);\n+\t\tstrbuf_addstr(&full_name, name);\n+\t\tname = full_name.buf;\n+\t}\n+\n+\t/*\n \t * With \"--full-name\", prefix_len=0; this caller needs to pass\n \t * an empty string in that case (a NULL is good for \"\").\n \t */\n@@ -152,55 +170,84 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/**\n+ * Recursively call ls-files on a submodule\n+ */\n+static void show_gitlink(const struct cache_entry *ce)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tint status;\n+\n+\targv_array_pushf(&cp.args, \"--submodule-prefix=%s%s/\",\n+\t\t\t submodule_prefix ? submodule_prefix : \"\",\n+\t\t\t ce->name);\n+\targv_array_push(&cp.args, \"ls-files\");\n+\targv_array_push(&cp.args, \"--recurse-submodules\");\n+\n+\tcp.git_cmd = 1;\n+\tcp.dir = ce->name;\n+\tstatus = run_command(&cp);\n+\tif (status)\n+\t\texit(status);\n+}\n+\n static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n {\n+\tstruct strbuf name = STRBUF_INIT;\n \tint len = max_prefix_len;\n+\tif (submodule_prefix)\n+\t\tstrbuf_addstr(&name, submodule_prefix);\n+\tstrbuf_addstr(&name, ce->name);\n \n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (!match_pathspec(&pathspec, ce->name, ce_namelen(ce),\n-\t\t\t len, ps_matched,\n-\t\t\t S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)))\n-\t\treturn;\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\t\tshow_gitlink(ce);\n+\t} else if (match_pathspec(&pathspec, name.buf, name.len,\n+\t\t\t\t len, ps_matched,\n+\t\t\t\t S_ISDIR(ce->ce_mode) ||\n+\t\t\t\t S_ISGITLINK(ce->ce_mode))) {\n+\t\tif (tag && *tag && show_valid_bit &&\n+\t\t (ce->ce_flags & CE_VALID)) {\n+\t\t\tstatic char alttag[4];\n+\t\t\tmemcpy(alttag, tag, 3);\n+\t\t\tif (isalpha(tag[0]))\n+\t\t\t\talttag[0] = tolower(tag[0]);\n+\t\t\telse if (tag[0] == '?')\n+\t\t\t\talttag[0] = '!';\n+\t\t\telse {\n+\t\t\t\talttag[0] = 'v';\n+\t\t\t\talttag[1] = tag[0];\n+\t\t\t\talttag[2] = ' ';\n+\t\t\t\talttag[3] = 0;\n+\t\t\t}\n+\t\t\ttag = alttag;\n+\t\t}\n \n-\tif (tag && *tag && show_valid_bit &&\n-\t (ce->ce_flags & CE_VALID)) {\n-\t\tstatic char alttag[4];\n-\t\tmemcpy(alttag, tag, 3);\n-\t\tif (isalpha(tag[0]))\n-\t\t\talttag[0] = tolower(tag[0]);\n-\t\telse if (tag[0] == '?')\n-\t\t\talttag[0] = '!';\n-\t\telse {\n-\t\t\talttag[0] = 'v';\n-\t\t\talttag[1] = tag[0];\n-\t\t\talttag[2] = ' ';\n-\t\t\talttag[3] = 0;\n+\t\tif (!show_stage) {\n+\t\t\tfputs(tag, stdout);\n+\t\t} else {\n+\t\t\tprintf(\"%s%06o %s %d\\t\",\n+\t\t\t tag,\n+\t\t\t ce->ce_mode,\n+\t\t\t find_unique_abbrev(ce->sha1,abbrev),\n+\t\t\t ce_stage(ce));\n+\t\t}\n+\t\twrite_eolinfo(ce, ce->name);\n+\t\twrite_name(ce->name);\n+\t\tif (debug_mode) {\n+\t\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n+\n+\t\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n+\t\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n+\t\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n+\t\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n+\t\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n \t\t}\n-\t\ttag = alttag;\n \t}\n \n-\tif (!show_stage) {\n-\t\tfputs(tag, stdout);\n-\t} else {\n-\t\tprintf(\"%s%06o %s %d\\t\",\n-\t\t tag,\n-\t\t ce->ce_mode,\n-\t\t find_unique_abbrev(ce->sha1,abbrev),\n-\t\t ce_stage(ce));\n-\t}\n-\twrite_eolinfo(ce, ce->name);\n-\twrite_name(ce->name);\n-\tif (debug_mode) {\n-\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n-\n-\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n-\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n-\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n-\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n-\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n-\t}\n+\tstrbuf_release(&name);\n }\n \n static void show_ru_info(void)\n@@ -468,6 +515,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\t{ OPTION_SET_INT, 0, \"full-name\", &prefix_len, NULL,\n \t\t\tN_(\"make the output relative to the project top directory\"),\n \t\t\tPARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL },\n+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\tN_(\"recurse through submodules\")),\n \t\tOPT_BOOL(0, \"error-unmatch\", &error_unmatch,\n \t\t\tN_(\"if any is not in the index, treat this as an error\")),\n \t\tOPT_STRING(0, \"with-tree\", &with_tree, N_(\"tree-ish\"),\n@@ -519,6 +568,24 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules)\n+\t\tsubmodule_prefix = getenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT);\n+\n+\tif (recurse_submodules &&\n+\t (show_stage || show_deleted || show_others || show_unmerged ||\n+\t show_killed || show_modified || show_resolve_undo ||\n+\t show_valid_bit || show_tag || show_eol || with_tree ||\n+\t (line_terminator == '\\0')))\n+\t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n+\n+\tif (recurse_submodules && error_unmatch)\n+\t\tdie(\"ls-files --recurse-submodules does not support \"\n+\t\t \"--error-unmatch\");\n+\n+\tif (recurse_submodules && argc)\n+\t\tdie(\"ls-files --recurse-submodules does not support path \"\n+\t\t \"arguments\");\n+\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\ndiff --git a/git.c b/git.c\nindex b2b096a..510b6a2 100644\n--- a/git.c\n+++ b/git.c\n@@ -440,7 +440,7 @@ static struct cmd_struct commands[] = {\n \t{ \"init-db\", cmd_init_db },\n \t{ \"interpret-trailers\", cmd_interpret_trailers, RUN_SETUP_GENTLY },\n \t{ \"log\", cmd_log, RUN_SETUP },\n-\t{ \"ls-files\", cmd_ls_files, RUN_SETUP },\n+\t{ \"ls-files\", cmd_ls_files, RUN_SETUP | SUPPORT_SUBMODULES },\n \t{ \"ls-remote\", cmd_ls_remote, RUN_SETUP_GENTLY },\n \t{ \"ls-tree\", cmd_ls_tree, RUN_SETUP },\n \t{ \"mailinfo\", cmd_mailinfo },\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nnew file mode 100755\nindex 0000000..7d225ac\n--- /dev/null\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -0,0 +1,100 @@\n+#!/bin/sh\n+\n+test_description='Test ls-files recurse-submodules feature\n+\n+This test verifies the recurse-submodules feature correctly lists files from\n+submodules.\n+'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup directory structure and submodules' '\n+\techo a >a &&\n+\tmkdir b &&\n+\techo b >b/b &&\n+\tgit add a b &&\n+\tgit commit -m \"add a and b\" &&\n+\tgit init submodule &&\n+\techo c >submodule/c &&\n+\tgit -C submodule add c &&\n+\tgit -C submodule commit -m \"add c\" &&\n+\tgit submodule add ./submodule &&\n+\tgit commit -m \"added submodule\"\n+'\n+\n+test_expect_success 'ls-files correctly outputs files in submodule' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files does not output files not added to a repo' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\techo a >not_added &&\n+\techo b >b/not_added &&\n+\techo c >submodule/not_added &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files recurses more than 1 level' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/subsub/d\n+\tEOF\n+\n+\tgit init submodule/subsub &&\n+\techo d >submodule/subsub/d &&\n+\tgit -C submodule/subsub add d &&\n+\tgit -C submodule/subsub commit -m \"add d\" &&\n+\tgit -C submodule submodule add ./subsub &&\n+\tgit -C submodule commit -m \"added subsub\" &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support using path arguments' '\n+\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n+\ttest_i18ngrep \"does not support path arguments\" actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support --error-unmatch' '\n+\ttest_must_fail git ls-files --recurse-submodules --error-unmatch 2>actual &&\n+\ttest_i18ngrep \"does not support --error-unmatch\" actual\n+'\n+\n+test_incompatible_with_recurse_submodules () {\n+\ttest_expect_success \"--recurse-submodules and $1 are incompatible\" \"\n+\t\ttest_must_fail git ls-files --recurse-submodules $1 2>actual &&\n+\t\ttest_i18ngrep 'unsupported mode' actual\n+\t\"\n+}\n+\n+test_incompatible_with_recurse_submodules -z\n+test_incompatible_with_recurse_submodules -v\n+test_incompatible_with_recurse_submodules -t\n+test_incompatible_with_recurse_submodules --deleted\n+test_incompatible_with_recurse_submodules --modified\n+test_incompatible_with_recurse_submodules --others\n+test_incompatible_with_recurse_submodules --stage\n+test_incompatible_with_recurse_submodules --killed\n+test_incompatible_with_recurse_submodules --unmerged\n+test_incompatible_with_recurse_submodules --eol\n+\n+test_done\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302658","messageId":"1474930003-83750-5-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474930003-83750-1-git-send-email-bmwill@google.com","subject":"[PATCH 4/4 v4] ls-files: add pathspec matching for submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-26T22:46:43Z","receivedAt":"2016-09-26T22:47:25Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pathspecs can be a bit tricky when trying to apply them to submodules.\nThe main challenge is that the pathspecs will be with respect to the\nsuperproject and not with respect to paths in the submodule. The\napproach this patch takes is to pass in the identical pathspec from the\nsuperproject to the submodule in addition to the submodule-prefix, which\nis the path from the root of the superproject to the submodule, and then\nwe can compare an entry in the submodule prepended with the\nsubmodule-prefix to the pathspec in order to determine if there is a\nmatch.\n\nThis patch also permits the pathspec logic to perform a prefix match against\nsubmodules since a pathspec could refer to a file inside of a submodule.\nDue to limitations in the wildmatch logic, a prefix match is only done\nliterally. If any wildcard character is encountered we'll simply punt\nand produce a false positive match. More accurate matching will be done\nonce inside the submodule. This is due to the superproject not knowing\nwhat files could exist in the submodule.\n\nSigned-off-by: Brandon Williams \n---\n builtin/ls-files.c | 28 ++++++---\n dir.c | 46 +++++++++++++-\n dir.h | 4 ++\n t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++-\n 4 files changed, 174 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex a39367f..2488a02 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -199,6 +199,7 @@ static void show_gitlink(const struct cache_entry *ce)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tint status;\n+\tint i;\n \n \targv_array_pushf(&cp.args, \"--submodule-prefix=%s%s/\",\n \t\t\t submodule_prefix ? submodule_prefix : \"\",\n@@ -209,6 +210,15 @@ static void show_gitlink(const struct cache_entry *ce)\n \t/* add supported options */\n \targv_array_pushv(&cp.args, recurse_submodules_opts.argv);\n \n+\t/*\n+\t * Pass in the original pathspec args. The submodule will be\n+\t * responsible for prepending the 'submodule_prefix' prior to comparing\n+\t * against the pathspec for matches.\n+\t */\n+\targv_array_push(&cp.args, \"--\");\n+\tfor (i = 0; i < pathspec.nr; i++)\n+\t\targv_array_push(&cp.args, pathspec.items[i].original);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -227,7 +237,8 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&\n+\t submodule_path_match(&pathspec, name.buf, ps_matched)) {\n \t\tshow_gitlink(ce);\n \t} else if (match_pathspec(&pathspec, name.buf, name.len,\n \t\t\t\t len, ps_matched,\n@@ -608,17 +619,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\tdie(\"ls-files --recurse-submodules does not support \"\n \t\t \"--error-unmatch\");\n \n-\tif (recurse_submodules && argc)\n-\t\tdie(\"ls-files --recurse-submodules does not support path \"\n-\t\t \"arguments\");\n-\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\n \t\t prefix, argv);\n \n-\t/* Find common prefix for all pathspec's */\n-\tmax_prefix = common_prefix(&pathspec);\n+\t/*\n+\t * Find common prefix for all pathspec's\n+\t * This is used as a performance optimization which unfortunately cannot\n+\t * be done when recursing into submodules\n+\t */\n+\tif (recurse_submodules)\n+\t\tmax_prefix = NULL;\n+\telse\n+\t\tmax_prefix = common_prefix(&pathspec);\n \tmax_prefix_len = max_prefix ? strlen(max_prefix) : 0;\n \n \t/* Treat unmatching pathspec elements as errors */\ndiff --git a/dir.c b/dir.c\nindex 0ea235f..28e9736 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -207,8 +207,9 @@ int within_depth(const char *name, int namelen,\n \treturn 1;\n }\n \n-#define DO_MATCH_EXCLUDE 1\n-#define DO_MATCH_DIRECTORY 2\n+#define DO_MATCH_EXCLUDE (1<<0)\n+#define DO_MATCH_DIRECTORY (1<<1)\n+#define DO_MATCH_SUBMODULE (1<<2)\n \n /*\n * Does 'match' match the given name?\n@@ -283,6 +284,32 @@ static int match_pathspec_item(const struct pathspec_item *item, int prefix,\n \t\t\t item->nowildcard_len - prefix))\n \t\treturn MATCHED_FNMATCH;\n \n+\t/* Perform checks to see if \"name\" is a super set of the pathspec */\n+\tif (flags & DO_MATCH_SUBMODULE) {\n+\t\t/* name is a literal prefix of the pathspec */\n+\t\tif ((namelen < matchlen) &&\n+\t\t (match[namelen] == '/') &&\n+\t\t !ps_strncmp(item, match, name, namelen))\n+\t\t\treturn MATCHED_RECURSIVELY;\n+\n+\t\t/* name\" doesn't match up to the first wild character */\n+\t\tif (item->nowildcard_len < item->len &&\n+\t\t ps_strncmp(item, match, name,\n+\t\t\t item->nowildcard_len - prefix))\n+\t\t\treturn 0;\n+\n+\t\t/*\n+\t\t * Here is where we would perform a wildmatch to check if\n+\t\t * \"name\" can be matched as a directory (or a prefix) against\n+\t\t * the pathspec. Since wildmatch doesn't have this capability\n+\t\t * at the present we have to punt and say that it is a match,\n+\t\t * potentially returning a false positive\n+\t\t * The submodules themselves will be able to perform more\n+\t\t * accurate matching to determine if the pathspec matches.\n+\t\t */\n+\t\treturn MATCHED_RECURSIVELY;\n+\t}\n+\n \treturn 0;\n }\n \n@@ -386,6 +413,21 @@ int match_pathspec(const struct pathspec *ps,\n \treturn negative ? 0 : positive;\n }\n \n+/**\n+ * Check if a submodule is a superset of the pathspec\n+ */\n+int submodule_path_match(const struct pathspec *ps,\n+\t\t\t const char *submodule_name,\n+\t\t\t char *seen)\n+{\n+\tint matched = do_match_pathspec(ps, submodule_name,\n+\t\t\t\t\tstrlen(submodule_name),\n+\t\t\t\t\t0, seen,\n+\t\t\t\t\tDO_MATCH_DIRECTORY |\n+\t\t\t\t\tDO_MATCH_SUBMODULE);\n+\treturn matched;\n+}\n+\n int report_path_error(const char *ps_matched,\n \t\t const struct pathspec *pathspec,\n \t\t const char *prefix)\ndiff --git a/dir.h b/dir.h\nindex da1a858..97c83bb 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item,\n \t\t const char *pattern, const char *string,\n \t\t int prefix);\n \n+extern int submodule_path_match(const struct pathspec *ps,\n+\t\t\t\tconst char *submodule_name,\n+\t\t\t\tchar *seen);\n+\n static inline int ce_path_match(const struct cache_entry *ce,\n \t\t\t\tconst struct pathspec *pathspec,\n \t\t\t\tchar *seen)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex 40767da..4a51d38 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -81,9 +81,111 @@ test_expect_success 'ls-files recurses more than 1 level' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success '--recurse-submodules does not support using path arguments' '\n-\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n-\ttest_i18ngrep \"does not support path arguments\" actual\n+test_expect_success '--recurse-submodules and pathspecs setup' '\n+\techo e >submodule/subsub/e.txt &&\n+\tgit -C submodule/subsub add e.txt &&\n+\tgit -C submodule/subsub commit -m \"adding e.txt\" &&\n+\techo f >submodule/f.TXT &&\n+\techo g >submodule/g.txt &&\n+\tgit -C submodule add f.TXT g.txt &&\n+\tgit -C submodule commit -m \"add f and g\" &&\n+\techo h >h.txt &&\n+\tmkdir sib &&\n+\techo sib >sib/file &&\n+\tgit add h.txt sib/file &&\n+\tgit commit -m \"add h and sib/file\" &&\n+\tgit init sub &&\n+\techo sub >sub/file &&\n+\tgit -C sub add file &&\n+\tgit -C sub commit -m \"add file\" &&\n+\tgit submodule add ./sub &&\n+\tgit commit -m \"added sub\" &&\n+\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\th.txt\n+\tsib/file\n+\tsub/file\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/d\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual &&\n+\tcat actual &&\n+\tgit ls-files --recurse-submodules \"*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" \":(exclude)submodule/subsub/*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"sub\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su*/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su?/file\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsib/file\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"s??/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s???file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s*file\" >actual &&\n+\ttest_cmp expect actual\n '\n \n test_expect_success '--recurse-submodules does not support --error-unmatch' '\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302659","messageId":"1474930003-83750-4-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474930003-83750-1-git-send-email-bmwill@google.com","subject":"[PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-26T22:46:42Z","receivedAt":"2016-09-26T22:47:30Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pass through some known-safe options when recursing into submodules.\n(--cached, --stage, -v, -t, -z, --debug, --eol)\n\nSigned-off-by: Brandon Williams \n---\n builtin/ls-files.c | 34 ++++++++++++++++++++++++++++++----\n t/t3007-ls-files-recurse-submodules.sh | 17 ++++++++++++-----\n 2 files changed, 42 insertions(+), 9 deletions(-)\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex d4bfc60..a39367f 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -31,6 +31,7 @@ static int debug_mode;\n static int show_eol;\n static int recurse_submodules;\n static const char *submodule_prefix;\n+static struct argv_array recurse_submodules_opts = ARGV_ARRAY_INIT;\n \n static const char *prefix;\n static int max_prefix_len;\n@@ -170,6 +171,27 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/*\n+ * Compile an argv_array with all of the options supported by --recurse_submodules\n+ */\n+static void compile_submodule_options(int show_tag)\n+{\n+\tif (show_cached)\n+\t\targv_array_push(&recurse_submodules_opts, \"--cached\");\n+\tif (show_stage)\n+\t\targv_array_push(&recurse_submodules_opts, \"--stage\");\n+\tif (show_valid_bit)\n+\t\targv_array_push(&recurse_submodules_opts, \"-v\");\n+\tif (show_tag)\n+\t\targv_array_push(&recurse_submodules_opts, \"-t\");\n+\tif (line_terminator == '\\0')\n+\t\targv_array_push(&recurse_submodules_opts, \"-z\");\n+\tif (debug_mode)\n+\t\targv_array_push(&recurse_submodules_opts, \"--debug\");\n+\tif (show_eol)\n+\t\targv_array_push(&recurse_submodules_opts, \"--eol\");\n+}\n+\n /**\n * Recursively call ls-files on a submodule\n */\n@@ -184,6 +206,9 @@ static void show_gitlink(const struct cache_entry *ce)\n \targv_array_push(&cp.args, \"ls-files\");\n \targv_array_push(&cp.args, \"--recurse-submodules\");\n \n+\t/* add supported options */\n+\targv_array_pushv(&cp.args, recurse_submodules_opts.argv);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -568,14 +593,15 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n-\tif (recurse_submodules)\n+\tif (recurse_submodules) {\n \t\tsubmodule_prefix = getenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT);\n+\t\tcompile_submodule_options(show_tag);\n+\t}\n \n \tif (recurse_submodules &&\n-\t (show_stage || show_deleted || show_others || show_unmerged ||\n+\t (show_deleted || show_others || show_unmerged ||\n \t show_killed || show_modified || show_resolve_undo ||\n-\t show_valid_bit || show_tag || show_eol || with_tree ||\n-\t (line_terminator == '\\0')))\n+\t with_tree))\n \t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n \n \tif (recurse_submodules && error_unmatch)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex 7d225ac..40767da 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'ls-files correctly outputs files in submodule with -z' '\n+\tcat | tr \"\\n\" \"\\0\" >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules -z >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'ls-files does not output files not added to a repo' '\n \tcat >expect <<-\\EOF &&\n \t.gitmodules\n@@ -86,15 +98,10 @@ test_incompatible_with_recurse_submodules () {\n \t\"\n }\n \n-test_incompatible_with_recurse_submodules -z\n-test_incompatible_with_recurse_submodules -v\n-test_incompatible_with_recurse_submodules -t\n test_incompatible_with_recurse_submodules --deleted\n test_incompatible_with_recurse_submodules --modified\n test_incompatible_with_recurse_submodules --others\n-test_incompatible_with_recurse_submodules --stage\n test_incompatible_with_recurse_submodules --killed\n test_incompatible_with_recurse_submodules --unmerged\n-test_incompatible_with_recurse_submodules --eol\n \n test_done\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302731","messageId":"xmqqtwd1nr56.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1474930003-83750-2-git-send-email-bmwill@google.com","subject":"Re: [PATCH 1/4 v4] submodules: make submodule-prefix option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T18:17:57Z","receivedAt":"2016-09-27T18:18:11Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> +--submodule-prefix=::\n> +\tSet a prefix which gives submodules context about the superproject that\n> +\tinvoked it. Only allowed for commands which support submodules.\n\nThis, and also the message in die(), uses a phrase \"support\nsubmodules\", but it is unclear what it exactly means to the end\nusers and readers.\n\nA \"ls-files\" that is recursively run as an implementation detail of\nthe \"grep --recurse-submodules\" would be taught to support this\noption with this series. Who is supporting submodules in that\ncontext?\n\nI'd imagine (close to) 100% of the people would say it is \"grep\"\nthat is supporting submodules, not \"ls-files\", but what this\nparagraph and die() message want to express by the phrase \"support\nsubmodules\" is the fact that \"ls-files\" knows how to react to\n\"--submodule-prefix\" option.\n\nI'd suggest not to worry too much about this phrasing at this point,\nuntil we figure out exactly how we want to present these to end\nusers. For now, perhaps drop the second sentence and replace it\nwith \"The end-users are not expected to use this option\" or\nsomething like that?\n\n> diff --git a/git.c b/git.c\n> index 1c61151..b2b096a 100644\n> --- a/git.c\n> +++ b/git.c\n> @@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n> \t\t\tsetenv(GIT_WORK_TREE_ENVIRONMENT, cmd, 1);\n> \t\t\tif (envchanged)\n> \t\t\t\t*envchanged = 1;\n> +\t\t} else if (!strcmp(cmd, \"--submodule-prefix\")) {\n> +\t\t\tif (*argc < 2) {\n> +\t\t\t\tfprintf(stderr, \"No prefix given for --submodule-prefix.\\n\" );\n> +\t\t\t\tusage(git_usage_string);\n> +\t\t\t}\n> +\t\t\tsetenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT, (*argv)[1], 1);\n> +\t\t\tif (envchanged)\n> +\t\t\t\t*envchanged = 1;\n> +\t\t\t(*argv)++;\n> +\t\t\t(*argc)--;\n> +\t\t} else if (skip_prefix(cmd, \"--submodule-prefix=\", &cmd)) {\n> +\t\t\tsetenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT, cmd, 1);\n> +\t\t\tif (envchanged)\n> +\t\t\t\t*envchanged = 1;\n> \t\t} else if (!strcmp(cmd, \"--bare\")) {\n> \t\t\tchar *cwd = xgetcwd();\n> \t\t\tis_bare_repository_cfg = 1;\n> @@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)\n> * RUN_SETUP for reading from the configuration file.\n> */\n> #define NEED_WORK_TREE\t\t(1<<3)\n> +#define SUPPORT_SUBMODULES\t(1<<4)\n> \n> struct cmd_struct {\n> \tconst char *cmd;\n> @@ -344,6 +359,10 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n> \t}\n> \tcommit_pager_choice();\n> \n> +\tif (!help && (getenv(GIT_SUBMODULE_PREFIX_ENVIRONMENT) &&\n> +\t\t !(p->option & SUPPORT_SUBMODULES)))\n> +\t\tdie(\"%s doesn't support submodules\", p->cmd);\n\ns/submodules/submodule-prefix/ at least.\n\n> \tif (!help && p->option & NEED_WORK_TREE)\n> \t\tsetup_work_tree();\n"},{"id":"302734","messageId":"xmqqlgydnqlh.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1474930003-83750-3-git-send-email-bmwill@google.com","subject":"Re: [PATCH 2/4 v4] ls-files: optionally recurse into submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T18:29:46Z","receivedAt":"2016-09-27T18:29:54Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> Allow ls-files to recognize submodules in order to retrieve a list of\n> files from a repository's submodules. This is done by forking off a\n> process to recursively call ls-files on all submodules. Use top-level\n> --submodule_prefix option to pass a path to the submodule which it can\n> use to prepend to output or pathspec matching logic.\n>\n> Signed-off-by: Brandon Williams \n> ---\n> Documentation/git-ls-files.txt | 7 +-\n> builtin/ls-files.c | 143 ++++++++++++++++++++++++---------\n> git.c | 2 +-\n> t/t3007-ls-files-recurse-submodules.sh | 100 +++++++++++++++++++++++\n> 4 files changed, 212 insertions(+), 40 deletions(-)\n> create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n>\n> diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\n> index 0d933ac..446209e 100644\n> --- a/Documentation/git-ls-files.txt\n> +++ b/Documentation/git-ls-files.txt\n> @@ -18,7 +18,8 @@ SYNOPSIS\n> \t\t[--exclude-per-directory=]\n> \t\t[--exclude-standard]\n> \t\t[--error-unmatch] [--with-tree=]\n> -\t\t[--full-name] [--abbrev] [--] [...]\n> +\t\t[--full-name] [--recurse-submodules]\n> +\t\t[--abbrev] [--] [...]\n> \n> DESCRIPTION\n> -----------\n> @@ -137,6 +138,10 @@ a space) at the start of each line:\n> \toption forces paths to be output relative to the project\n> \ttop directory.\n> \n> +--recurse-submodules::\n> +\tRecursively calls ls-files on each submodule in the repository.\n> +\tCurrently there is only support for the --cached mode.\n> +\n> --abbrev[=]::\n> \tInstead of showing the full 40-byte hexadecimal object\n> \tlines, show only a partial prefix.\n> diff --git a/builtin/ls-files.c b/builtin/ls-files.c\n> index 00ea91a..d4bfc60 100644\n> --- a/builtin/ls-files.c\n> +++ b/builtin/ls-files.c\n> @@ -14,6 +14,7 @@\n> #include \"resolve-undo.h\"\n> #include \"string-list.h\"\n> #include \"pathspec.h\"\n> +#include \"run-command.h\"\n> \n> static int abbrev;\n> static int show_deleted;\n> @@ -28,6 +29,8 @@ static int show_valid_bit;\n> static int line_terminator = '\\n';\n> static int debug_mode;\n> static int show_eol;\n> +static int recurse_submodules;\n> +static const char *submodule_prefix;\n\nI would have expected this to added to environment.c in the previous\nstep, but it is OK--I'd imagine you'd grab this from the environment\nand carrying a piece of information from git.c to here by setenv()\nfollowed by getenv() feels somewhat roundabout, though.\n\n> static const char *prefix;\n> static int max_prefix_len;\n> @@ -68,6 +71,21 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)\n> static void write_name(const char *name)\n> {\n> \t/*\n> +\t * NEEDSWORK: To make this thread-safe, full_name would have to be owned\n> +\t * by the caller.\n\nAs Peff mentioned in his review in another thread, a large number of\nfunctions in git are not reentrant, and I do not think we would want\nto give the impression that those missing a warning are safe to use.\n\nOther than that, this step looks OK. 3/4 and later would be a lot\nmore fun to review ;-)\n"},{"id":"302737","messageId":"xmqqh991nq34.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1474930003-83750-4-git-send-email-bmwill@google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T18:40:47Z","receivedAt":"2016-09-27T18:41:08Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> Pass through some known-safe options when recursing into submodules.\n> (--cached, --stage, -v, -t, -z, --debug, --eol)\n>\n> Signed-off-by: Brandon Williams \n> ---\n> builtin/ls-files.c | 34 ++++++++++++++++++++++++++++++----\n> t/t3007-ls-files-recurse-submodules.sh | 17 ++++++++++++-----\n> 2 files changed, 42 insertions(+), 9 deletions(-)\n>\n> diff --git a/builtin/ls-files.c b/builtin/ls-files.c\n> index d4bfc60..a39367f 100644\n> --- a/builtin/ls-files.c\n> +++ b/builtin/ls-files.c\n> @@ -31,6 +31,7 @@ static int debug_mode;\n> static int show_eol;\n> static int recurse_submodules;\n> static const char *submodule_prefix;\n> +static struct argv_array recurse_submodules_opts = ARGV_ARRAY_INIT;\n\nI'd imagine that this is also thread-unsafe, but we do not have to\ncomment it ;-)\n\n> @@ -170,6 +171,27 @@ static void show_killed_files(struct dir_struct *dir)\n> \t}\n> }\n> \n> +/*\n> + * Compile an argv_array with all of the options supported by --recurse_submodules\n> + */\n> +static void compile_submodule_options(int show_tag)\n> +{\n> +\tif (show_cached)\n> +\t\targv_array_push(&recurse_submodules_opts, \"--cached\");\n> +\tif (show_stage)\n> +\t\targv_array_push(&recurse_submodules_opts, \"--stage\");\n> +\tif (show_valid_bit)\n> +\t\targv_array_push(&recurse_submodules_opts, \"-v\");\n> +\tif (show_tag)\n> +\t\targv_array_push(&recurse_submodules_opts, \"-t\");\n> +\tif (line_terminator == '\\0')\n> +\t\targv_array_push(&recurse_submodules_opts, \"-z\");\n> +\tif (debug_mode)\n> +\t\targv_array_push(&recurse_submodules_opts, \"--debug\");\n> +\tif (show_eol)\n> +\t\targv_array_push(&recurse_submodules_opts, \"--eol\");\n> +}\n\nOK. These are only the safe ones to pass through? \"compile\" or\n\"assemble\" is much less important thing to say than how these are\nchosen. \"pass_supported_options()\" or something? I dunno.\n\n> \tif (recurse_submodules &&\n> -\t (show_stage || show_deleted || show_others || show_unmerged ||\n> +\t (show_deleted || show_others || show_unmerged ||\n> \t show_killed || show_modified || show_resolve_undo ||\n> -\t show_valid_bit || show_tag || show_eol || with_tree ||\n> -\t (line_terminator == '\\0')))\n> +\t with_tree))\n> \t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n\nMakes sense.\n\n> +test_expect_success 'ls-files correctly outputs files in submodule with -z' '\n> +\tcat | tr \"\\n\" \"\\0\" >expect <<-\\EOF &&\n> +\t.gitmodules\n> +\ta\n> +\tb/b\n> +\tsubmodule/c\n> +\tEOF\n\nHmm, what do you need \"cat\" for here?\n\nIn nul_to_q and q_to_nul implementations (t/test-lib-functions.sh)\nwe seem to avoid using \"tr\", even though q_to_cr and others do use\nit. I wonder if we had some portability issues with passing NUL\nthrough tr or something?\n\n ... digs and finds e85fe4d8 (\"more tr portability test script\n fixes\", 2008-03-12)\n\nSo use something like\n\n\tperl -pe 'y/\\012/\\000/' <<\\-EOF\n ...\n EOF\n\ninstead, perhaps?\n\n> +\tgit ls-files --recurse-submodules -z >actual &&\n> +\ttest_cmp expect actual\n> +'\n"},{"id":"302738","messageId":"xmqqd1jpnpyh.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1474930003-83750-4-git-send-email-bmwill@google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T18:43:34Z","receivedAt":"2016-09-27T18:43:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> \tif (recurse_submodules &&\n> -\t (show_stage || show_deleted || show_others || show_unmerged ||\n> +\t (show_deleted || show_others || show_unmerged ||\n> \t show_killed || show_modified || show_resolve_undo ||\n> -\t show_valid_bit || show_tag || show_eol || with_tree ||\n> -\t (line_terminator == '\\0')))\n> +\t with_tree))\n> \t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n\nAhh, one more thing, and this comment probably applies to 2/4 not\nthis one, but if the intention is to shrink this \"not supported yet\"\ncheck as the series progresses, in the earlier step the check would\nneed to make sure no pathspec is given, which is first supported in\n4/4, I think. It is not a big deal to require rerolling by itself,\nbit if you are rerolling 2/4 for other reasons, then...\n"},{"id":"302750","messageId":"xmqq1t05nmcs.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1474930003-83750-5-git-send-email-bmwill@google.com","subject":"Re: [PATCH 4/4 v4] ls-files: add pathspec matching for submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T20:01:23Z","receivedAt":"2016-09-27T20:01:32Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> -\t/* Find common prefix for all pathspec's */\n> -\tmax_prefix = common_prefix(&pathspec);\n> +\t/*\n> +\t * Find common prefix for all pathspec's\n> +\t * This is used as a performance optimization which unfortunately cannot\n> +\t * be done when recursing into submodules\n> +\t */\n> +\tif (recurse_submodules)\n> +\t\tmax_prefix = NULL;\n> +\telse\n> +\t\tmax_prefix = common_prefix(&pathspec);\n> \tmax_prefix_len = max_prefix ? strlen(max_prefix) : 0;\n\nI still wonder if we can do better than this, as this would be a big\ncycle-saver especially in recurse-submodules case.\n\nWhen you get max_prefix that is \"a/b/c\", there are three cases:\n\n * a/b/c is a path prefix for an entry in the index, e.g. a/b/c/d;\n you then can safely use it and you do not have to do any\n recursive invocation of ls-files outside \"a/b/c\". You may match\n a/b/c/d in the toplevel, or you may recurse a/b/c/e that is a\n submodule, but you won't have to pay attention to submodules\n outside.\n\n * a leading path of a/b/c, e.g. a/b, is a gitlink or a blob in the\n index; you can use a/b and you only have to recurse into a/b if\n that is a submodule; if a/b is a blob, you'd show nothing.\n\n * a/b/c itself and no leading path of it appears in the index; you\n know that nothing will match once you know that you are in this\n situation.\n\nBecause a gitlink \"a/b\" sorts at the same location in the index as a\nregular blob \"a/b\" would, by feeding the max_prefix common_prefix()\ngives you (i.e. \"a/b/c\") to index_name_pos() to see which one of the\nthree situations you are in can be done fairly cheaply, I would\nthink. The index_name_pos() call may find \"a/b/c\" exactly (case 1),\nor return a location where \"a/b/c\" would be inserted in the list of\nexisting entries. If there were \"a/b\" (or \"a\") in the index, there\nwouldn't be any \"a/b/x\" (or \"a/x\") at the same time, so a query for\n\"a/b/c\" would land you next to (just after) an existing entry that\nis a leading path of it, if such an entry exists, no? That would\nallow you to tell case 2 above fairly cheaply, I would expect.\n\nIt is a separate issue if adding that support to 4/4 is a good idea;\nI personally think doing it as a separate follow-up patch would make\nmore sense, so all of the above is tangent.\n\n"},{"id":"302751","messageId":"xmqqwphxm7av.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"xmqqh991nq34.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T20:11:52Z","receivedAt":"2016-09-27T20:11:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Junio C Hamano writes:\n\n> In nul_to_q and q_to_nul implementations (t/test-lib-functions.sh)\n> we seem to avoid using \"tr\", even though q_to_cr and others do use\n> it. I wonder if we had some portability issues with passing NUL\n> through tr or something?\n>\n> ... digs and finds e85fe4d8 (\"more tr portability test script\n> fixes\", 2008-03-12)\n>\n> So use something like\n>\n> \tperl -pe 'y/\\012/\\000/' <<\\-EOF\n> ...\n> EOF\n>\n> instead, perhaps?\n\nI actually think it would make more sense to add\n\n lf_to_nul () {\n perl -pe 'y/\\012/\\000/'\n }\n\nto t/test-lib-functions.sh somewhere near q_to_nul if we were to go\nthis route.\n"},{"id":"302754","messageId":"20160927202925.GA32565@google.com","threadId":"44147","inReplyTo":"xmqqtwd1nr56.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 1/4 v4] submodules: make submodule-prefix option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-27T20:29:25Z","receivedAt":"2016-09-27T20:29:33Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/27, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > +--submodule-prefix=::\n> > +\tSet a prefix which gives submodules context about the superproject that\n> > +\tinvoked it. Only allowed for commands which support submodules.\n> \n> This, and also the message in die(), uses a phrase \"support\n> submodules\", but it is unclear what it exactly means to the end\n> users and readers.\n> \n> A \"ls-files\" that is recursively run as an implementation detail of\n> the \"grep --recurse-submodules\" would be taught to support this\n> option with this series. Who is supporting submodules in that\n> context?\n> \n> I'd imagine (close to) 100% of the people would say it is \"grep\"\n> that is supporting submodules, not \"ls-files\", but what this\n> paragraph and die() message want to express by the phrase \"support\n> submodules\" is the fact that \"ls-files\" knows how to react to\n> \"--submodule-prefix\" option.\n> \n> I'd suggest not to worry too much about this phrasing at this point,\n> until we figure out exactly how we want to present these to end\n> users. For now, perhaps drop the second sentence and replace it\n> with \"The end-users are not expected to use this option\" or\n> something like that?\n\nK can do. The intention is that each command has to do whatever\ninternal rework needed so that it understands how to interact with\nsubomodules (be that calling another command or internally supporting\nit).\n\n> > +\t\tdie(\"%s doesn't support submodules\", p->cmd);\n> \n> s/submodules/submodule-prefix/ at least.\n\nSo should the #define be SUPPORT_SUBMODULE_PREFIX instead? That may be\ntoo narrow minded and not looking toward future submodule options\nsupport but I'm not sure.\n\n-- \nBrandon Williams\n"},{"id":"302756","messageId":"20160927203338.GB32565@google.com","threadId":"44147","inReplyTo":"xmqqlgydnqlh.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 2/4 v4] ls-files: optionally recurse into submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-27T20:33:38Z","receivedAt":"2016-09-27T20:33:45Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/27, Junio C Hamano wrote:\n> Brandon Williams writes:\n> > +static const char *submodule_prefix;\n> \n> I would have expected this to added to environment.c in the previous\n> step, but it is OK--I'd imagine you'd grab this from the environment\n> and carrying a piece of information from git.c to here by setenv()\n> followed by getenv() feels somewhat roundabout, though.\n\nIf it would make sense to do the caching of prefix string in\nenvironment.c I can move it there and add a get_submodule_prefix()\nfunction which either reads the envvar or the cached value if its\nalready been read. Would that be a better route?\n\n> \n> > static const char *prefix;\n> > static int max_prefix_len;\n> > @@ -68,6 +71,21 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)\n> > static void write_name(const char *name)\n> > {\n> > \t/*\n> > +\t * NEEDSWORK: To make this thread-safe, full_name would have to be owned\n> > +\t * by the caller.\n> \n> As Peff mentioned in his review in another thread, a large number of\n> functions in git are not reentrant, and I do not think we would want\n> to give the impression that those missing a warning are safe to use.\n> \n> Other than that, this step looks OK. 3/4 and later would be a lot\n> more fun to review ;-)\n\nOh yes, I can remove the comment. Seemed to miss that bit while\nrerolling the series.\n\n-- \nBrandon Williams\n"},{"id":"302757","messageId":"xmqqoa39m67l.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"20160927202925.GA32565@google.com","subject":"Re: [PATCH 1/4 v4] submodules: make submodule-prefix option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T20:35:26Z","receivedAt":"2016-09-27T20:35:34Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n>> s/submodules/submodule-prefix/ at least.\n>\n> So should the #define be SUPPORT_SUBMODULE_PREFIX instead? That may be\n> too narrow minded and not looking toward future submodule options\n> support but I'm not sure.\n\nI am not convinced that this prefix needs to be tied/limited to\nsubmodule, at least not yet, though. I view it as a prefix that\npoints from above the repository's top, of which submodule support\nmay be a good sample user, but the caller may not necessarily be\ndoing or interested in \"submodule support\".\n\nThat's also part of figuring out how we want define the semantics of\nthis thing and how we want to present it to the end-users, I guess,\nso we may have to rename it when we know more, but that's OK.\n"},{"id":"302758","messageId":"20160927204020.GC32565@google.com","threadId":"44147","inReplyTo":"xmqq1t05nmcs.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 4/4 v4] ls-files: add pathspec matching for submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-27T20:40:20Z","receivedAt":"2016-09-27T20:40:28Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/27, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > -\t/* Find common prefix for all pathspec's */\n> > -\tmax_prefix = common_prefix(&pathspec);\n> > +\t/*\n> > +\t * Find common prefix for all pathspec's\n> > +\t * This is used as a performance optimization which unfortunately cannot\n> > +\t * be done when recursing into submodules\n> > +\t */\n> > +\tif (recurse_submodules)\n> > +\t\tmax_prefix = NULL;\n> > +\telse\n> > +\t\tmax_prefix = common_prefix(&pathspec);\n> > \tmax_prefix_len = max_prefix ? strlen(max_prefix) : 0;\n> \n> I still wonder if we can do better than this, as this would be a big\n> cycle-saver especially in recurse-submodules case.\n> \n> When you get max_prefix that is \"a/b/c\", there are three cases:\n> \n> * a/b/c is a path prefix for an entry in the index, e.g. a/b/c/d;\n> you then can safely use it and you do not have to do any\n> recursive invocation of ls-files outside \"a/b/c\". You may match\n> a/b/c/d in the toplevel, or you may recurse a/b/c/e that is a\n> submodule, but you won't have to pay attention to submodules\n> outside.\n> \n> * a leading path of a/b/c, e.g. a/b, is a gitlink or a blob in the\n> index; you can use a/b and you only have to recurse into a/b if\n> that is a submodule; if a/b is a blob, you'd show nothing.\n> \n> * a/b/c itself and no leading path of it appears in the index; you\n> know that nothing will match once you know that you are in this\n> situation.\n> \n> Because a gitlink \"a/b\" sorts at the same location in the index as a\n> regular blob \"a/b\" would, by feeding the max_prefix common_prefix()\n> gives you (i.e. \"a/b/c\") to index_name_pos() to see which one of the\n> three situations you are in can be done fairly cheaply, I would\n> think. The index_name_pos() call may find \"a/b/c\" exactly (case 1),\n> or return a location where \"a/b/c\" would be inserted in the list of\n> existing entries. If there were \"a/b\" (or \"a\") in the index, there\n> wouldn't be any \"a/b/x\" (or \"a/x\") at the same time, so a query for\n> \"a/b/c\" would land you next to (just after) an existing entry that\n> is a leading path of it, if such an entry exists, no? That would\n> allow you to tell case 2 above fairly cheaply, I would expect.\n> \n> It is a separate issue if adding that support to 4/4 is a good idea;\n> I personally think doing it as a separate follow-up patch would make\n> more sense, so all of the above is tangent.\n\nI agree that this could be a big cycle saver. At the present I was\nworking towards getting a working implementation but this should\ndefinitely be addressed in a follow-up patch to introduce the\noptimization to the recurse-submodule mode. It hopefully wouldn't be\ntoo hard to implement seeing as its using string literals.\n\n-- \nBrandon Williams\n"},{"id":"302759","messageId":"20160927204320.GD32565@google.com","threadId":"44147","inReplyTo":"xmqqoa39m67l.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 1/4 v4] submodules: make submodule-prefix option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-27T20:43:20Z","receivedAt":"2016-09-27T20:43:27Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/27, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> >> s/submodules/submodule-prefix/ at least.\n> >\n> > So should the #define be SUPPORT_SUBMODULE_PREFIX instead? That may be\n> > too narrow minded and not looking toward future submodule options\n> > support but I'm not sure.\n> \n> I am not convinced that this prefix needs to be tied/limited to\n> submodule, at least not yet, though. I view it as a prefix that\n> points from above the repository's top, of which submodule support\n> may be a good sample user, but the caller may not necessarily be\n> doing or interested in \"submodule support\".\n> \n> That's also part of figuring out how we want define the semantics of\n> this thing and how we want to present it to the end-users, I guess,\n> so we may have to rename it when we know more, but that's OK.\n\nK we can leave the name as is for now then. Thankfully it should be a\nsimple name change since it only lives in 1 file.\n\n-- \nBrandon Williams\n"},{"id":"302761","messageId":"20160927204440.GE32565@google.com","threadId":"44147","inReplyTo":"xmqqd1jpnpyh.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-27T20:44:40Z","receivedAt":"2016-09-27T20:44:48Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/27, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > \tif (recurse_submodules &&\n> > -\t (show_stage || show_deleted || show_others || show_unmerged ||\n> > +\t (show_deleted || show_others || show_unmerged ||\n> > \t show_killed || show_modified || show_resolve_undo ||\n> > -\t show_valid_bit || show_tag || show_eol || with_tree ||\n> > -\t (line_terminator == '\\0')))\n> > +\t with_tree))\n> > \t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n> \n> Ahh, one more thing, and this comment probably applies to 2/4 not\n> this one, but if the intention is to shrink this \"not supported yet\"\n> check as the series progresses, in the earlier step the check would\n> need to make sure no pathspec is given, which is first supported in\n> 4/4, I think. It is not a big deal to require rerolling by itself,\n> bit if you are rerolling 2/4 for other reasons, then...\n\nOh there is a separate if gaurd for pathspecs which is introduced in 2/4\nand then removed once pathspec support has been added in 4/4.\n\n-- \nBrandon Williams\n"},{"id":"302762","messageId":"20160927204935.GF32565@google.com","threadId":"44147","inReplyTo":"xmqqh991nq34.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-27T20:49:35Z","receivedAt":"2016-09-27T20:50:13Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/27, Junio C Hamano wrote:\n> Brandon Williams writes:\n> > @@ -170,6 +171,27 @@ static void show_killed_files(struct dir_struct *dir)\n> > \t}\n> > }\n> > \n> > +/*\n> > + * Compile an argv_array with all of the options supported by --recurse_submodules\n> > + */\n> > +static void compile_submodule_options(int show_tag)\n> > +{\n> > +\tif (show_cached)\n> > +\t\targv_array_push(&recurse_submodules_opts, \"--cached\");\n> > +\tif (show_stage)\n> > +\t\targv_array_push(&recurse_submodules_opts, \"--stage\");\n> > +\tif (show_valid_bit)\n> > +\t\targv_array_push(&recurse_submodules_opts, \"-v\");\n> > +\tif (show_tag)\n> > +\t\targv_array_push(&recurse_submodules_opts, \"-t\");\n> > +\tif (line_terminator == '\\0')\n> > +\t\targv_array_push(&recurse_submodules_opts, \"-z\");\n> > +\tif (debug_mode)\n> > +\t\targv_array_push(&recurse_submodules_opts, \"--debug\");\n> > +\tif (show_eol)\n> > +\t\targv_array_push(&recurse_submodules_opts, \"--eol\");\n> > +}\n> \n> OK. These are only the safe ones to pass through? \"compile\" or\n> \"assemble\" is much less important thing to say than how these are\n> chosen. \"pass_supported_options()\" or something? I dunno.\n\nAlternatively we can change this to compile all potential options (not\njust the ones that are supported now) and then just error the caller\nout, as you've suggested, if an unsupported option used. 'pass' may not\nbe the most descriptive word for this function as it isn't actually\ndoing the passing but rather generating an argv of options that will be\npassed in the event a submodule is found and we need to kick off a child\nprocess for it.\n\n-- \nBrandon Williams\n"},{"id":"302763","messageId":"20160927205202.GG32565@google.com","threadId":"44147","inReplyTo":"xmqqwphxm7av.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-27T20:52:02Z","receivedAt":"2016-09-27T20:53:17Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/27, Junio C Hamano wrote:\n> Junio C Hamano writes:\n> \n> > In nul_to_q and q_to_nul implementations (t/test-lib-functions.sh)\n> > we seem to avoid using \"tr\", even though q_to_cr and others do use\n> > it. I wonder if we had some portability issues with passing NUL\n> > through tr or something?\n> >\n> > ... digs and finds e85fe4d8 (\"more tr portability test script\n> > fixes\", 2008-03-12)\n> >\n> > So use something like\n> >\n> > \tperl -pe 'y/\\012/\\000/' <<\\-EOF\n> > ...\n> > EOF\n> >\n> > instead, perhaps?\n> \n> I actually think it would make more sense to add\n> \n> lf_to_nul () {\n> perl -pe 'y/\\012/\\000/'\n> }\n> \n> to t/test-lib-functions.sh somewhere near q_to_nul if we were to go\n> this route.\n\nmy mind is drawing a blank, what does the 'lf' in 'lf_to_nul' stand for?\nline feed?\n\n-- \nBrandon Williams\n"},{"id":"302765","messageId":"xmqqh991m55e.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"20160927205202.GG32565@google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T20:58:21Z","receivedAt":"2016-09-27T20:58:30Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> my mind is drawing a blank, what does the 'lf' in 'lf_to_nul' stand for?\n> line feed?\n\nYup. \"man 7 ascii\" ;-)\n"},{"id":"302766","messageId":"CAGZ79kZgchm1usviUWvLqrBC133mDuQwOMcay6k3L6Vsv-CPzA@mail.gmail.com","threadId":"44147","inReplyTo":"20160927205202.GG32565@google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-09-27T20:59:18Z","receivedAt":"2016-09-27T20:59:24Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Tue, Sep 27, 2016 at 1:52 PM, Brandon Williams wrote:\n> On 09/27, Junio C Hamano wrote:\n>> Junio C Hamano writes:\n>>\n>> > In nul_to_q and q_to_nul implementations (t/test-lib-functions.sh)\n>> > we seem to avoid using \"tr\", even though q_to_cr and others do use\n>> > it. I wonder if we had some portability issues with passing NUL\n>> > through tr or something?\n>> >\n>> > ... digs and finds e85fe4d8 (\"more tr portability test script\n>> > fixes\", 2008-03-12)\n>> >\n>> > So use something like\n>> >\n>> > perl -pe 'y/\\012/\\000/' <<\\-EOF\n>> > ...\n>> > EOF\n>> >\n>> > instead, perhaps?\n>>\n>> I actually think it would make more sense to add\n>>\n>> lf_to_nul () {\n>> perl -pe 'y/\\012/\\000/'\n>> }\n>>\n>> to t/test-lib-functions.sh somewhere near q_to_nul if we were to go\n>> this route.\n>\n> my mind is drawing a blank, what does the 'lf' in 'lf_to_nul' stand for?\n> line feed?\n>\n\nYes, line feed. (Note that Git has to deal with this cross platform new lines\ne.g. CRLF is common on Windows, CR was common on MAC, and LF is\nWindows, so naming the new line as they are, makes sense here.)\n"},{"id":"302770","messageId":"xmqqa8etm532.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"20160927204440.GE32565@google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-27T20:59:45Z","receivedAt":"2016-09-27T21:01:12Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> Oh there is a separate if gaurd for pathspecs which is introduced in 2/4\n> and then removed once pathspec support has been added in 4/4.\n\nThanks; I missed to spot that when I wrote the message you are\nresponding to, but it indeed is there ;-)\n"},{"id":"302816","messageId":"20160928172417.GA61176@google.com","threadId":"44147","inReplyTo":"xmqqwphxm7av.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-28T17:24:17Z","receivedAt":"2016-09-28T17:24:24Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/27, Junio C Hamano wrote:\n> Junio C Hamano writes:\n> \n> > In nul_to_q and q_to_nul implementations (t/test-lib-functions.sh)\n> > we seem to avoid using \"tr\", even though q_to_cr and others do use\n> > it. I wonder if we had some portability issues with passing NUL\n> > through tr or something?\n> >\n> > ... digs and finds e85fe4d8 (\"more tr portability test script\n> > fixes\", 2008-03-12)\n> >\n> > So use something like\n> >\n> > \tperl -pe 'y/\\012/\\000/' <<\\-EOF\n> > ...\n> > EOF\n> >\n> > instead, perhaps?\n> \n> I actually think it would make more sense to add\n> \n> lf_to_nul () {\n> perl -pe 'y/\\012/\\000/'\n> }\n> \n> to t/test-lib-functions.sh somewhere near q_to_nul if we were to go\n> this route.\n\nTurns out this function already exists in test-lib-functions.sh\n\n-- \nBrandon Williams\n"},{"id":"302827","messageId":"xmqq4m4zkfzy.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"20160928172417.GA61176@google.com","subject":"Re: [PATCH 3/4 v4] ls-files: pass through safe options for --recurse-submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-28T18:59:13Z","receivedAt":"2016-09-28T18:59:22Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n>> I actually think it would make more sense to add\n>> \n>> lf_to_nul () {\n>> perl -pe 'y/\\012/\\000/'\n>> }\n>> \n>> to t/test-lib-functions.sh somewhere near q_to_nul if we were to go\n>> this route.\n>\n> Turns out this function already exists in test-lib-functions.sh\n\n;-)\n"},{"id":"302842","messageId":"1475099443-145608-1-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1474930003-83750-1-git-send-email-bmwill@google.com","subject":"[PATCH v5 0/4] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-28T21:50:39Z","receivedAt":"2016-09-28T21:51:21Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"The big change in this version is the introduction of a --super-prefix option\nto the top level git. After much discussion this seemed to be a better naming\nscheme than 'submodule-prefix' as it could be an option other cmds could use\nindependent of submodules.\n\nIn 3/4 I changed the compile_submodule_options function to compile all options that can be realistically passed through and when an option that isn't supported (or rather safe) yet is provided the caller will be errored out.\n\nBrandon Williams (4):\n git: make super-prefix option\n ls-files: optionally recurse into submodules\n ls-files: pass through safe options for --recurse-submodules\n ls-files: add pathspec matching for submodules\n\n Documentation/git-ls-files.txt | 7 +-\n Documentation/git.txt | 6 +\n builtin/ls-files.c | 202 ++++++++++++++++++++++++-------\n cache.h | 2 +\n dir.c | 46 +++++++-\n dir.h | 4 +\n environment.c | 10 ++\n git.c | 24 +++-\n t/t3007-ls-files-recurse-submodules.sh | 209 +++++++++++++++++++++++++++++++++\n 9 files changed, 466 insertions(+), 44 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302843","messageId":"1475099443-145608-2-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475099443-145608-1-git-send-email-bmwill@google.com","subject":"[PATCH v5 1/4] git: make super-prefix option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-28T21:50:40Z","receivedAt":"2016-09-28T21:51:26Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX'\nwhich can be used to specify a path from above a repository down to its\nroot. The immediate use of this option is by commands which have a\n--recurse-submodule option in order to give context to submodules about\nhow they were invoked. This option is currently only allowed for\nbuiltins which support a super-prefix.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git.txt | 6 ++++++\n cache.h | 2 ++\n environment.c | 10 ++++++++++\n git.c | 22 ++++++++++++++++++++++\n 4 files changed, 40 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 7913fc2..e3309b9 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -13,6 +13,7 @@ SYNOPSIS\n [--exec-path[=]] [--html-path] [--man-path] [--info-path]\n [-p|--paginate|--no-pager] [--no-replace-objects] [--bare]\n [--git-dir=] [--work-tree=] [--namespace=]\n+ [--super-prefix=]\n []\n \n DESCRIPTION\n@@ -601,6 +602,11 @@ foo.bar= ...`) sets `foo.bar` to the empty string.\n \tdetails. Equivalent to setting the `GIT_NAMESPACE` environment\n \tvariable.\n \n+--super-prefix=::\n+\tSet a prefix which gives a path from above a repository down to its\n+\troot. One use is to give submodules context about the superproject that\n+\tinvoked it. Currently for internal use only.\n+\n --bare::\n \tTreat the repository as a bare repository. If GIT_DIR\n \tenvironment is not set, it is set to the current working\ndiff --git a/cache.h b/cache.h\nindex 3556326..01730e1 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)\n #define GIT_NAMESPACE_ENVIRONMENT \"GIT_NAMESPACE\"\n #define GIT_WORK_TREE_ENVIRONMENT \"GIT_WORK_TREE\"\n #define GIT_PREFIX_ENVIRONMENT \"GIT_PREFIX\"\n+#define GIT_SUPER_PREFIX_ENVIRONMENT \"GIT_INTERNAL_SUPER_PREFIX\"\n #define DEFAULT_GIT_DIR_ENVIRONMENT \".git\"\n #define DB_ENVIRONMENT \"GIT_OBJECT_DIRECTORY\"\n #define INDEX_ENVIRONMENT \"GIT_INDEX_FILE\"\n@@ -468,6 +469,7 @@ extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);\n extern int get_common_dir(struct strbuf *sb, const char *gitdir);\n extern const char *get_git_namespace(void);\n extern const char *strip_namespace(const char *namespaced_ref);\n+extern const char *get_super_prefix();\n extern const char *get_git_work_tree(void);\n \n /*\ndiff --git a/environment.c b/environment.c\nindex ca72464..1a2a779 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -100,6 +100,8 @@ static char *work_tree;\n static const char *namespace;\n static size_t namespace_len;\n \n+static const char *super_prefix;\n+\n static const char *git_dir, *git_common_dir;\n static char *git_object_dir, *git_index_file, *git_graft_file;\n int git_db_env, git_index_env, git_graft_env, git_common_dir_env;\n@@ -120,6 +122,7 @@ const char * const local_repo_env[] = {\n \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n \tGIT_PREFIX_ENVIRONMENT,\n+\tGIT_SUPER_PREFIX_ENVIRONMENT,\n \tGIT_SHALLOW_FILE_ENVIRONMENT,\n \tGIT_COMMON_DIR_ENVIRONMENT,\n \tNULL\n@@ -222,6 +225,13 @@ const char *strip_namespace(const char *namespaced_ref)\n \treturn namespaced_ref + namespace_len;\n }\n \n+const char *get_super_prefix()\n+{\n+\tif (!super_prefix)\n+\t\tsuper_prefix = getenv(GIT_SUPER_PREFIX_ENVIRONMENT);\n+\treturn super_prefix;\n+}\n+\n static int git_work_tree_initialized;\n \n /*\ndiff --git a/git.c b/git.c\nindex 1c61151..3e5cd63 100644\n--- a/git.c\n+++ b/git.c\n@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n \t\t\tsetenv(GIT_WORK_TREE_ENVIRONMENT, cmd, 1);\n \t\t\tif (envchanged)\n \t\t\t\t*envchanged = 1;\n+\t\t} else if (!strcmp(cmd, \"--super-prefix\")) {\n+\t\t\tif (*argc < 2) {\n+\t\t\t\tfprintf(stderr, \"No prefix given for --super-prefix.\\n\" );\n+\t\t\t\tusage(git_usage_string);\n+\t\t\t}\n+\t\t\tsetenv(GIT_SUPER_PREFIX_ENVIRONMENT, (*argv)[1], 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n+\t\t\t(*argv)++;\n+\t\t\t(*argc)--;\n+\t\t} else if (skip_prefix(cmd, \"--super-prefix=\", &cmd)) {\n+\t\t\tsetenv(GIT_SUPER_PREFIX_ENVIRONMENT, cmd, 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--bare\")) {\n \t\t\tchar *cwd = xgetcwd();\n \t\t\tis_bare_repository_cfg = 1;\n@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)\n * RUN_SETUP for reading from the configuration file.\n */\n #define NEED_WORK_TREE\t\t(1<<3)\n+#define SUPPORT_SUPER_PREFIX\t(1<<4)\n \n struct cmd_struct {\n \tconst char *cmd;\n@@ -344,6 +359,13 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n \t}\n \tcommit_pager_choice();\n \n+\tif (!help && get_super_prefix()) {\n+\t\tif (!(p->option & SUPPORT_SUPER_PREFIX))\n+\t\t\tdie(\"%s doesn't support --super-prefix\", p->cmd);\n+\t\tif (prefix)\n+\t\t\tdie(\"can't have both a prefix and super-prefix\");\n+\t}\n+\n \tif (!help && p->option & NEED_WORK_TREE)\n \t\tsetup_work_tree();\n \n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302844","messageId":"1475099443-145608-3-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475099443-145608-1-git-send-email-bmwill@google.com","subject":"[PATCH v5 2/4] ls-files: optionally recurse into submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-28T21:50:41Z","receivedAt":"2016-09-28T21:51:32Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Allow ls-files to recognize submodules in order to retrieve a list of\nfiles from a repository's submodules. This is done by forking off a\nprocess to recursively call ls-files on all submodules. Use top-level\n--super-prefix option to pass a path to the submodule which it can\nuse to prepend to output or pathspec matching logic.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git-ls-files.txt | 7 +-\n builtin/ls-files.c | 139 ++++++++++++++++++++++++---------\n git.c | 2 +-\n t/t3007-ls-files-recurse-submodules.sh | 100 ++++++++++++++++++++++++\n 4 files changed, 208 insertions(+), 40 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\ndiff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\nindex 0d933ac..446209e 100644\n--- a/Documentation/git-ls-files.txt\n+++ b/Documentation/git-ls-files.txt\n@@ -18,7 +18,8 @@ SYNOPSIS\n \t\t[--exclude-per-directory=]\n \t\t[--exclude-standard]\n \t\t[--error-unmatch] [--with-tree=]\n-\t\t[--full-name] [--abbrev] [--] [...]\n+\t\t[--full-name] [--recurse-submodules]\n+\t\t[--abbrev] [--] [...]\n \n DESCRIPTION\n -----------\n@@ -137,6 +138,10 @@ a space) at the start of each line:\n \toption forces paths to be output relative to the project\n \ttop directory.\n \n+--recurse-submodules::\n+\tRecursively calls ls-files on each submodule in the repository.\n+\tCurrently there is only support for the --cached mode.\n+\n --abbrev[=]::\n \tInstead of showing the full 40-byte hexadecimal object\n \tlines, show only a partial prefix.\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 00ea91a..e0e5cf5 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -14,6 +14,7 @@\n #include \"resolve-undo.h\"\n #include \"string-list.h\"\n #include \"pathspec.h\"\n+#include \"run-command.h\"\n \n static int abbrev;\n static int show_deleted;\n@@ -28,8 +29,10 @@ static int show_valid_bit;\n static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n+static int recurse_submodules;\n \n static const char *prefix;\n+static const char *super_prefix;\n static int max_prefix_len;\n static int prefix_len;\n static struct pathspec pathspec;\n@@ -68,6 +71,19 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)\n static void write_name(const char *name)\n {\n \t/*\n+\t * Prepend the super_prefix to name to construct the full_name to be\n+\t * written. 'full_name' gets reused across output lines to minimize the\n+\t * allocation churn.\n+\t */\n+\tstatic struct strbuf full_name = STRBUF_INIT;\n+\tif (super_prefix && *super_prefix) {\n+\t\tstrbuf_reset(&full_name);\n+\t\tstrbuf_addstr(&full_name, super_prefix);\n+\t\tstrbuf_addstr(&full_name, name);\n+\t\tname = full_name.buf;\n+\t}\n+\n+\t/*\n \t * With \"--full-name\", prefix_len=0; this caller needs to pass\n \t * an empty string in that case (a NULL is good for \"\").\n \t */\n@@ -152,55 +168,84 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/**\n+ * Recursively call ls-files on a submodule\n+ */\n+static void show_gitlink(const struct cache_entry *ce)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tint status;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n+\t\t\t super_prefix ? super_prefix : \"\",\n+\t\t\t ce->name);\n+\targv_array_push(&cp.args, \"ls-files\");\n+\targv_array_push(&cp.args, \"--recurse-submodules\");\n+\n+\tcp.git_cmd = 1;\n+\tcp.dir = ce->name;\n+\tstatus = run_command(&cp);\n+\tif (status)\n+\t\texit(status);\n+}\n+\n static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n {\n+\tstruct strbuf name = STRBUF_INIT;\n \tint len = max_prefix_len;\n+\tif (super_prefix)\n+\t\tstrbuf_addstr(&name, super_prefix);\n+\tstrbuf_addstr(&name, ce->name);\n \n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (!match_pathspec(&pathspec, ce->name, ce_namelen(ce),\n-\t\t\t len, ps_matched,\n-\t\t\t S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)))\n-\t\treturn;\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\t\tshow_gitlink(ce);\n+\t} else if (match_pathspec(&pathspec, name.buf, name.len,\n+\t\t\t\t len, ps_matched,\n+\t\t\t\t S_ISDIR(ce->ce_mode) ||\n+\t\t\t\t S_ISGITLINK(ce->ce_mode))) {\n+\t\tif (tag && *tag && show_valid_bit &&\n+\t\t (ce->ce_flags & CE_VALID)) {\n+\t\t\tstatic char alttag[4];\n+\t\t\tmemcpy(alttag, tag, 3);\n+\t\t\tif (isalpha(tag[0]))\n+\t\t\t\talttag[0] = tolower(tag[0]);\n+\t\t\telse if (tag[0] == '?')\n+\t\t\t\talttag[0] = '!';\n+\t\t\telse {\n+\t\t\t\talttag[0] = 'v';\n+\t\t\t\talttag[1] = tag[0];\n+\t\t\t\talttag[2] = ' ';\n+\t\t\t\talttag[3] = 0;\n+\t\t\t}\n+\t\t\ttag = alttag;\n+\t\t}\n \n-\tif (tag && *tag && show_valid_bit &&\n-\t (ce->ce_flags & CE_VALID)) {\n-\t\tstatic char alttag[4];\n-\t\tmemcpy(alttag, tag, 3);\n-\t\tif (isalpha(tag[0]))\n-\t\t\talttag[0] = tolower(tag[0]);\n-\t\telse if (tag[0] == '?')\n-\t\t\talttag[0] = '!';\n-\t\telse {\n-\t\t\talttag[0] = 'v';\n-\t\t\talttag[1] = tag[0];\n-\t\t\talttag[2] = ' ';\n-\t\t\talttag[3] = 0;\n+\t\tif (!show_stage) {\n+\t\t\tfputs(tag, stdout);\n+\t\t} else {\n+\t\t\tprintf(\"%s%06o %s %d\\t\",\n+\t\t\t tag,\n+\t\t\t ce->ce_mode,\n+\t\t\t find_unique_abbrev(ce->sha1,abbrev),\n+\t\t\t ce_stage(ce));\n+\t\t}\n+\t\twrite_eolinfo(ce, ce->name);\n+\t\twrite_name(ce->name);\n+\t\tif (debug_mode) {\n+\t\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n+\n+\t\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n+\t\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n+\t\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n+\t\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n+\t\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n \t\t}\n-\t\ttag = alttag;\n \t}\n \n-\tif (!show_stage) {\n-\t\tfputs(tag, stdout);\n-\t} else {\n-\t\tprintf(\"%s%06o %s %d\\t\",\n-\t\t tag,\n-\t\t ce->ce_mode,\n-\t\t find_unique_abbrev(ce->sha1,abbrev),\n-\t\t ce_stage(ce));\n-\t}\n-\twrite_eolinfo(ce, ce->name);\n-\twrite_name(ce->name);\n-\tif (debug_mode) {\n-\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n-\n-\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n-\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n-\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n-\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n-\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n-\t}\n+\tstrbuf_release(&name);\n }\n \n static void show_ru_info(void)\n@@ -468,6 +513,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\t{ OPTION_SET_INT, 0, \"full-name\", &prefix_len, NULL,\n \t\t\tN_(\"make the output relative to the project top directory\"),\n \t\t\tPARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL },\n+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\tN_(\"recurse through submodules\")),\n \t\tOPT_BOOL(0, \"error-unmatch\", &error_unmatch,\n \t\t\tN_(\"if any is not in the index, treat this as an error\")),\n \t\tOPT_STRING(0, \"with-tree\", &with_tree, N_(\"tree-ish\"),\n@@ -484,6 +531,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tprefix = cmd_prefix;\n \tif (prefix)\n \t\tprefix_len = strlen(prefix);\n+\tsuper_prefix = get_super_prefix();\n \tgit_config(git_default_config, NULL);\n \n \tif (read_cache() < 0)\n@@ -519,6 +567,21 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules &&\n+\t (show_stage || show_deleted || show_others || show_unmerged ||\n+\t show_killed || show_modified || show_resolve_undo ||\n+\t show_valid_bit || show_tag || show_eol || with_tree ||\n+\t (line_terminator == '\\0')))\n+\t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n+\n+\tif (recurse_submodules && error_unmatch)\n+\t\tdie(\"ls-files --recurse-submodules does not support \"\n+\t\t \"--error-unmatch\");\n+\n+\tif (recurse_submodules && argc)\n+\t\tdie(\"ls-files --recurse-submodules does not support path \"\n+\t\t \"arguments\");\n+\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\ndiff --git a/git.c b/git.c\nindex 3e5cd63..86d9be4 100644\n--- a/git.c\n+++ b/git.c\n@@ -443,7 +443,7 @@ static struct cmd_struct commands[] = {\n \t{ \"init-db\", cmd_init_db },\n \t{ \"interpret-trailers\", cmd_interpret_trailers, RUN_SETUP_GENTLY },\n \t{ \"log\", cmd_log, RUN_SETUP },\n-\t{ \"ls-files\", cmd_ls_files, RUN_SETUP },\n+\t{ \"ls-files\", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },\n \t{ \"ls-remote\", cmd_ls_remote, RUN_SETUP_GENTLY },\n \t{ \"ls-tree\", cmd_ls_tree, RUN_SETUP },\n \t{ \"mailinfo\", cmd_mailinfo },\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nnew file mode 100755\nindex 0000000..7d225ac\n--- /dev/null\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -0,0 +1,100 @@\n+#!/bin/sh\n+\n+test_description='Test ls-files recurse-submodules feature\n+\n+This test verifies the recurse-submodules feature correctly lists files from\n+submodules.\n+'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup directory structure and submodules' '\n+\techo a >a &&\n+\tmkdir b &&\n+\techo b >b/b &&\n+\tgit add a b &&\n+\tgit commit -m \"add a and b\" &&\n+\tgit init submodule &&\n+\techo c >submodule/c &&\n+\tgit -C submodule add c &&\n+\tgit -C submodule commit -m \"add c\" &&\n+\tgit submodule add ./submodule &&\n+\tgit commit -m \"added submodule\"\n+'\n+\n+test_expect_success 'ls-files correctly outputs files in submodule' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files does not output files not added to a repo' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\techo a >not_added &&\n+\techo b >b/not_added &&\n+\techo c >submodule/not_added &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files recurses more than 1 level' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/subsub/d\n+\tEOF\n+\n+\tgit init submodule/subsub &&\n+\techo d >submodule/subsub/d &&\n+\tgit -C submodule/subsub add d &&\n+\tgit -C submodule/subsub commit -m \"add d\" &&\n+\tgit -C submodule submodule add ./subsub &&\n+\tgit -C submodule commit -m \"added subsub\" &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support using path arguments' '\n+\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n+\ttest_i18ngrep \"does not support path arguments\" actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support --error-unmatch' '\n+\ttest_must_fail git ls-files --recurse-submodules --error-unmatch 2>actual &&\n+\ttest_i18ngrep \"does not support --error-unmatch\" actual\n+'\n+\n+test_incompatible_with_recurse_submodules () {\n+\ttest_expect_success \"--recurse-submodules and $1 are incompatible\" \"\n+\t\ttest_must_fail git ls-files --recurse-submodules $1 2>actual &&\n+\t\ttest_i18ngrep 'unsupported mode' actual\n+\t\"\n+}\n+\n+test_incompatible_with_recurse_submodules -z\n+test_incompatible_with_recurse_submodules -v\n+test_incompatible_with_recurse_submodules -t\n+test_incompatible_with_recurse_submodules --deleted\n+test_incompatible_with_recurse_submodules --modified\n+test_incompatible_with_recurse_submodules --others\n+test_incompatible_with_recurse_submodules --stage\n+test_incompatible_with_recurse_submodules --killed\n+test_incompatible_with_recurse_submodules --unmerged\n+test_incompatible_with_recurse_submodules --eol\n+\n+test_done\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302845","messageId":"1475099443-145608-4-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475099443-145608-1-git-send-email-bmwill@google.com","subject":"[PATCH v5 3/4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-28T21:50:42Z","receivedAt":"2016-09-28T21:51:35Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pass through some known-safe options when recursing into submodules.\n(--cached, --stage, -v, -t, -z, --debug, --eol)\n\nOther options are compiled into an argv_array but if an unsafe option is\ngiven the caller will be errored out.\n\nSigned-off-by: Brandon Williams \n---\n builtin/ls-files.c | 51 ++++++++++++++++++++++++++++++++--\n t/t3007-ls-files-recurse-submodules.sh | 17 ++++++++----\n 2 files changed, 60 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex e0e5cf5..f377e36 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -30,6 +30,7 @@ static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n static int recurse_submodules;\n+static struct argv_array submodules_options = ARGV_ARRAY_INIT;\n \n static const char *prefix;\n static const char *super_prefix;\n@@ -168,6 +169,45 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/*\n+ * Compile an argv_array with all of the options supported by --recurse_submodules\n+ */\n+static void compile_submodule_options(const struct dir_struct *dir, int show_tag)\n+{\n+\tif (line_terminator == '\\0')\n+\t\targv_array_push(&submodules_options, \"-z\");\n+\tif (show_tag)\n+\t\targv_array_push(&submodules_options, \"-t\");\n+\tif (show_valid_bit)\n+\t\targv_array_push(&submodules_options, \"-v\");\n+\tif (show_cached)\n+\t\targv_array_push(&submodules_options, \"--cached\");\n+\tif (show_deleted)\n+\t\targv_array_push(&submodules_options, \"--deleted\");\n+\tif (show_modified)\n+\t\targv_array_push(&submodules_options, \"--modified\");\n+\tif (show_others)\n+\t\targv_array_push(&submodules_options, \"--others\");\n+\tif (dir->flags & DIR_SHOW_IGNORED)\n+\t\targv_array_push(&submodules_options, \"--ignored\");\n+\tif (show_stage)\n+\t\targv_array_push(&submodules_options, \"--stage\");\n+\tif (show_killed)\n+\t\targv_array_push(&submodules_options, \"--killed\");\n+\tif (dir->flags & DIR_SHOW_OTHER_DIRECTORIES)\n+\t\targv_array_push(&submodules_options, \"--directory\");\n+\tif (!(dir->flags & DIR_SHOW_OTHER_DIRECTORIES))\n+\t\targv_array_push(&submodules_options, \"--empty-directory\");\n+\tif (show_unmerged)\n+\t\targv_array_push(&submodules_options, \"--unmerged\");\n+\tif (show_resolve_undo)\n+\t\targv_array_push(&submodules_options, \"--resolve-undo\");\n+\tif (show_eol)\n+\t\targv_array_push(&submodules_options, \"--eol\");\n+\tif (debug_mode)\n+\t\targv_array_push(&submodules_options, \"--debug\");\n+}\n+\n /**\n * Recursively call ls-files on a submodule\n */\n@@ -182,6 +222,9 @@ static void show_gitlink(const struct cache_entry *ce)\n \targv_array_push(&cp.args, \"ls-files\");\n \targv_array_push(&cp.args, \"--recurse-submodules\");\n \n+\t/* add supported options */\n+\targv_array_pushv(&cp.args, submodules_options.argv);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -567,11 +610,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules)\n+\t\tcompile_submodule_options(&dir, show_tag);\n+\n \tif (recurse_submodules &&\n-\t (show_stage || show_deleted || show_others || show_unmerged ||\n+\t (show_deleted || show_others || show_unmerged ||\n \t show_killed || show_modified || show_resolve_undo ||\n-\t show_valid_bit || show_tag || show_eol || with_tree ||\n-\t (line_terminator == '\\0')))\n+\t with_tree))\n \t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n \n \tif (recurse_submodules && error_unmatch)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex 7d225ac..79107d8 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'ls-files correctly outputs files in submodule with -z' '\n+\tlf_to_nul >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules -z >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'ls-files does not output files not added to a repo' '\n \tcat >expect <<-\\EOF &&\n \t.gitmodules\n@@ -86,15 +98,10 @@ test_incompatible_with_recurse_submodules () {\n \t\"\n }\n \n-test_incompatible_with_recurse_submodules -z\n-test_incompatible_with_recurse_submodules -v\n-test_incompatible_with_recurse_submodules -t\n test_incompatible_with_recurse_submodules --deleted\n test_incompatible_with_recurse_submodules --modified\n test_incompatible_with_recurse_submodules --others\n-test_incompatible_with_recurse_submodules --stage\n test_incompatible_with_recurse_submodules --killed\n test_incompatible_with_recurse_submodules --unmerged\n-test_incompatible_with_recurse_submodules --eol\n \n test_done\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302846","messageId":"1475099443-145608-5-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475099443-145608-1-git-send-email-bmwill@google.com","subject":"[PATCH v5 4/4] ls-files: add pathspec matching for submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-28T21:50:43Z","receivedAt":"2016-09-28T21:51:38Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pathspecs can be a bit tricky when trying to apply them to submodules.\nThe main challenge is that the pathspecs will be with respect to the\nsuperproject and not with respect to paths in the submodule. The\napproach this patch takes is to pass in the identical pathspec from the\nsuperproject to the submodule in addition to the submodule-prefix, which\nis the path from the root of the superproject to the submodule, and then\nwe can compare an entry in the submodule prepended with the\nsubmodule-prefix to the pathspec in order to determine if there is a\nmatch.\n\nThis patch also permits the pathspec logic to perform a prefix match against\nsubmodules since a pathspec could refer to a file inside of a submodule.\nDue to limitations in the wildmatch logic, a prefix match is only done\nliterally. If any wildcard character is encountered we'll simply punt\nand produce a false positive match. More accurate matching will be done\nonce inside the submodule. This is due to the superproject not knowing\nwhat files could exist in the submodule.\n\nSigned-off-by: Brandon Williams \n---\n builtin/ls-files.c | 28 ++++++---\n dir.c | 46 +++++++++++++-\n dir.h | 4 ++\n t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++-\n 4 files changed, 174 insertions(+), 12 deletions(-)\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex f377e36..c8c5c29 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -215,6 +215,7 @@ static void show_gitlink(const struct cache_entry *ce)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tint status;\n+\tint i;\n \n \targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n \t\t\t super_prefix ? super_prefix : \"\",\n@@ -225,6 +226,15 @@ static void show_gitlink(const struct cache_entry *ce)\n \t/* add supported options */\n \targv_array_pushv(&cp.args, submodules_options.argv);\n \n+\t/*\n+\t * Pass in the original pathspec args. The submodule will be\n+\t * responsible for prepending the 'submodule_prefix' prior to comparing\n+\t * against the pathspec for matches.\n+\t */\n+\targv_array_push(&cp.args, \"--\");\n+\tfor (i = 0; i < pathspec.nr; i++)\n+\t\targv_array_push(&cp.args, pathspec.items[i].original);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -243,7 +253,8 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&\n+\t submodule_path_match(&pathspec, name.buf, ps_matched)) {\n \t\tshow_gitlink(ce);\n \t} else if (match_pathspec(&pathspec, name.buf, name.len,\n \t\t\t\t len, ps_matched,\n@@ -623,17 +634,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\tdie(\"ls-files --recurse-submodules does not support \"\n \t\t \"--error-unmatch\");\n \n-\tif (recurse_submodules && argc)\n-\t\tdie(\"ls-files --recurse-submodules does not support path \"\n-\t\t \"arguments\");\n-\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\n \t\t prefix, argv);\n \n-\t/* Find common prefix for all pathspec's */\n-\tmax_prefix = common_prefix(&pathspec);\n+\t/*\n+\t * Find common prefix for all pathspec's\n+\t * This is used as a performance optimization which unfortunately cannot\n+\t * be done when recursing into submodules\n+\t */\n+\tif (recurse_submodules)\n+\t\tmax_prefix = NULL;\n+\telse\n+\t\tmax_prefix = common_prefix(&pathspec);\n \tmax_prefix_len = max_prefix ? strlen(max_prefix) : 0;\n \n \t/* Treat unmatching pathspec elements as errors */\ndiff --git a/dir.c b/dir.c\nindex 0ea235f..28e9736 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -207,8 +207,9 @@ int within_depth(const char *name, int namelen,\n \treturn 1;\n }\n \n-#define DO_MATCH_EXCLUDE 1\n-#define DO_MATCH_DIRECTORY 2\n+#define DO_MATCH_EXCLUDE (1<<0)\n+#define DO_MATCH_DIRECTORY (1<<1)\n+#define DO_MATCH_SUBMODULE (1<<2)\n \n /*\n * Does 'match' match the given name?\n@@ -283,6 +284,32 @@ static int match_pathspec_item(const struct pathspec_item *item, int prefix,\n \t\t\t item->nowildcard_len - prefix))\n \t\treturn MATCHED_FNMATCH;\n \n+\t/* Perform checks to see if \"name\" is a super set of the pathspec */\n+\tif (flags & DO_MATCH_SUBMODULE) {\n+\t\t/* name is a literal prefix of the pathspec */\n+\t\tif ((namelen < matchlen) &&\n+\t\t (match[namelen] == '/') &&\n+\t\t !ps_strncmp(item, match, name, namelen))\n+\t\t\treturn MATCHED_RECURSIVELY;\n+\n+\t\t/* name\" doesn't match up to the first wild character */\n+\t\tif (item->nowildcard_len < item->len &&\n+\t\t ps_strncmp(item, match, name,\n+\t\t\t item->nowildcard_len - prefix))\n+\t\t\treturn 0;\n+\n+\t\t/*\n+\t\t * Here is where we would perform a wildmatch to check if\n+\t\t * \"name\" can be matched as a directory (or a prefix) against\n+\t\t * the pathspec. Since wildmatch doesn't have this capability\n+\t\t * at the present we have to punt and say that it is a match,\n+\t\t * potentially returning a false positive\n+\t\t * The submodules themselves will be able to perform more\n+\t\t * accurate matching to determine if the pathspec matches.\n+\t\t */\n+\t\treturn MATCHED_RECURSIVELY;\n+\t}\n+\n \treturn 0;\n }\n \n@@ -386,6 +413,21 @@ int match_pathspec(const struct pathspec *ps,\n \treturn negative ? 0 : positive;\n }\n \n+/**\n+ * Check if a submodule is a superset of the pathspec\n+ */\n+int submodule_path_match(const struct pathspec *ps,\n+\t\t\t const char *submodule_name,\n+\t\t\t char *seen)\n+{\n+\tint matched = do_match_pathspec(ps, submodule_name,\n+\t\t\t\t\tstrlen(submodule_name),\n+\t\t\t\t\t0, seen,\n+\t\t\t\t\tDO_MATCH_DIRECTORY |\n+\t\t\t\t\tDO_MATCH_SUBMODULE);\n+\treturn matched;\n+}\n+\n int report_path_error(const char *ps_matched,\n \t\t const struct pathspec *pathspec,\n \t\t const char *prefix)\ndiff --git a/dir.h b/dir.h\nindex da1a858..97c83bb 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item,\n \t\t const char *pattern, const char *string,\n \t\t int prefix);\n \n+extern int submodule_path_match(const struct pathspec *ps,\n+\t\t\t\tconst char *submodule_name,\n+\t\t\t\tchar *seen);\n+\n static inline int ce_path_match(const struct cache_entry *ce,\n \t\t\t\tconst struct pathspec *pathspec,\n \t\t\t\tchar *seen)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex 79107d8..5475855 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -81,9 +81,111 @@ test_expect_success 'ls-files recurses more than 1 level' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success '--recurse-submodules does not support using path arguments' '\n-\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n-\ttest_i18ngrep \"does not support path arguments\" actual\n+test_expect_success '--recurse-submodules and pathspecs setup' '\n+\techo e >submodule/subsub/e.txt &&\n+\tgit -C submodule/subsub add e.txt &&\n+\tgit -C submodule/subsub commit -m \"adding e.txt\" &&\n+\techo f >submodule/f.TXT &&\n+\techo g >submodule/g.txt &&\n+\tgit -C submodule add f.TXT g.txt &&\n+\tgit -C submodule commit -m \"add f and g\" &&\n+\techo h >h.txt &&\n+\tmkdir sib &&\n+\techo sib >sib/file &&\n+\tgit add h.txt sib/file &&\n+\tgit commit -m \"add h and sib/file\" &&\n+\tgit init sub &&\n+\techo sub >sub/file &&\n+\tgit -C sub add file &&\n+\tgit -C sub commit -m \"add file\" &&\n+\tgit submodule add ./sub &&\n+\tgit commit -m \"added sub\" &&\n+\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\th.txt\n+\tsib/file\n+\tsub/file\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/d\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual &&\n+\tcat actual &&\n+\tgit ls-files --recurse-submodules \"*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" \":(exclude)submodule/subsub/*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"sub\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su*/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su?/file\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsib/file\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"s??/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s???file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s*file\" >actual &&\n+\ttest_cmp expect actual\n '\n \n test_expect_success '--recurse-submodules does not support --error-unmatch' '\n-- \n2.8.0.rc3.226.g39d4020\n\n"},{"id":"302848","messageId":"CAGZ79kZRg34pG2+NUU4qz=Jxxe=6+fdPD+szGv-QTC4vs4DuMg@mail.gmail.com","threadId":"44147","inReplyTo":"1475099443-145608-2-git-send-email-bmwill@google.com","subject":"Re: [PATCH v5 1/4] git: make super-prefix option","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-09-28T22:01:29Z","receivedAt":"2016-09-28T22:01:35Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Sep 28, 2016 at 2:50 PM, Brandon Williams wrote:\n\n>\n> DESCRIPTION\n> @@ -601,6 +602,11 @@ foo.bar= ...`) sets `foo.bar` to the empty string.\n> details. Equivalent to setting the `GIT_NAMESPACE` environment\n> variable.\n>\n> +--super-prefix=::\n> + Set a prefix which gives a path from above a repository down to its\n> + root. One use is to give submodules context about the superproject that\n> + invoked it. Currently for internal use only.\n\nI would put the \"Currently for internal use only.\" at the beginning, so end\nusers don't have to bother reading though the description, when they want to\nonly use kosher flags. (Well, checking `man git fetch` and searching\nfor 'internal',\nthere doesn't seem to be a consistent way how to document internal flags :(.\nIt doesn't however advertise the flag in the SYNOPSIS. Ok it doesn't advertise\na lot of flags in its SYNOPSIS)\n\n> +const char *get_super_prefix()\n> +{\n> + if (!super_prefix)\n> + super_prefix = getenv(GIT_SUPER_PREFIX_ENVIRONMENT);\n\nThe getenv() function returns a pointer to the value in the\nenvironment, or NULL if there is no match.\n\nSo in case this is not set (when e.g. the user did not specify the\nsuper prefix),\nwe would probe it a couple of times.\n\nThe caching effect only occurs when the string is set. So this looks\nlike we save repetitive\ncalls, but we do not always do that.\n"},{"id":"302849","messageId":"CAGZ79kZy5LrXeq_w1oNzXnMsQdBaWqbqR0Dh6FHL39vmg7gZQQ@mail.gmail.com","threadId":"44147","inReplyTo":"1475099443-145608-3-git-send-email-bmwill@google.com","subject":"Re: [PATCH v5 2/4] ls-files: optionally recurse into submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-09-28T22:11:23Z","receivedAt":"2016-09-28T22:11:29Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Wed, Sep 28, 2016 at 2:50 PM, Brandon Williams wrote:\n> Allow ls-files to recognize submodules in order to retrieve a list of\n> files from a repository's submodules. This is done by forking off a\n> process to recursively call ls-files on all submodules. Use top-level\n> --super-prefix option to pass a path to the submodule which it can\n> use to prepend to output or pathspec matching logic.\n>\n> Signed-off-by: Brandon Williams \n> ---\n> Documentation/git-ls-files.txt | 7 +-\n> builtin/ls-files.c | 139 ++++++++++++++++++++++++---------\n> git.c | 2 +-\n> t/t3007-ls-files-recurse-submodules.sh | 100 ++++++++++++++++++++++++\n> 4 files changed, 208 insertions(+), 40 deletions(-)\n> create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n>\n> diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\n> index 0d933ac..446209e 100644\n> --- a/Documentation/git-ls-files.txt\n> +++ b/Documentation/git-ls-files.txt\n> @@ -18,7 +18,8 @@ SYNOPSIS\n> [--exclude-per-directory=]\n> [--exclude-standard]\n> [--error-unmatch] [--with-tree=]\n> - [--full-name] [--abbrev] [--] [...]\n> + [--full-name] [--recurse-submodules]\n> + [--abbrev] [--] [...]\n>\n> DESCRIPTION\n> -----------\n> @@ -137,6 +138,10 @@ a space) at the start of each line:\n> option forces paths to be output relative to the project\n> top directory.\n>\n> +--recurse-submodules::\n> + Recursively calls ls-files on each submodule in the repository.\n> + Currently there is only support for the --cached mode.\n> +\n> --abbrev[=]::\n> Instead of showing the full 40-byte hexadecimal object\n> lines, show only a partial prefix.\n> diff --git a/builtin/ls-files.c b/builtin/ls-files.c\n> index 00ea91a..e0e5cf5 100644\n> --- a/builtin/ls-files.c\n> +++ b/builtin/ls-files.c\n> @@ -14,6 +14,7 @@\n> #include \"resolve-undo.h\"\n> #include \"string-list.h\"\n> #include \"pathspec.h\"\n> +#include \"run-command.h\"\n>\n> static int abbrev;\n> static int show_deleted;\n> @@ -28,8 +29,10 @@ static int show_valid_bit;\n> static int line_terminator = '\\n';\n> static int debug_mode;\n> static int show_eol;\n> +static int recurse_submodules;\n>\n> static const char *prefix;\n> +static const char *super_prefix;\n> static int max_prefix_len;\n> static int prefix_len;\n> static struct pathspec pathspec;\n> @@ -68,6 +71,19 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)\n> static void write_name(const char *name)\n> {\n> /*\n> + * Prepend the super_prefix to name to construct the full_name to be\n> + * written. 'full_name' gets reused across output lines to minimize the\n> + * allocation churn.\n> + */\n\nWhen doing these tricks with the allocation churn (i.e. we make it\nhard to read and understand\nfor a reader, then we should do it completely, i.e. keep full_name in\nthe strbuf and\nonly do a strbuf_setlen to reset the buffer just a bit. With this\nimplementation we burden the\nreader/user to understand how the memory is kept over multiple calls\nto this function,\nbut we still do more work than expected). So either I'd not worry\nabout performance\nand provide an 'obvious correct' implementation, with e.g. no static\nhere and we free the memory\ncorrectly. Or you'd go the performance route, but then we'd usually\nask for numbers.\n(How much faster is it; Does the trickyness trade off well to the\nperformance gain?)\n\n\n> + static struct strbuf full_name = STRBUF_INIT;\n> + if (super_prefix && *super_prefix) {\n\nWhy do we have to check twice here? Wouldn't just\n\n if (super_prefix) {\n ...\n\nbe enough?\n"},{"id":"302850","messageId":"xmqqzimrhdlh.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1475099443-145608-2-git-send-email-bmwill@google.com","subject":"Re: [PATCH v5 1/4] git: make super-prefix option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-28T22:19:22Z","receivedAt":"2016-09-28T22:19:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX'\n> which can be used to specify a path from above a repository down to its\n> root. The immediate use of this option is by commands which have a\n> --recurse-submodule option in order to give context to submodules about\n> how they were invoked. This option is currently only allowed for\n> builtins which support a super-prefix.\n\nYes, it can be used to specify that, but it is unclear (1) how such\na path thusly specified is used, (2) how it affects the outcome of\nthe operations, and (3) what it means (the same applies to the\ndocumentation update).\n\nIt would help future readers of \"git log\", \"tig blame\", etc. to have\na few sentences after the first sentence above. Perhaps\n\n ... specify a path from above ... down to its root. When such a\n super-prefix is specified, the paths reported by Git are\n prefixed with it to make them relative to that directory\n \"above\". The paths given by the users on the command line\n (e.g. \"git subcmd --output-file=path/to/a/file\" and pathspecs)\n are taken relative to that directory \"above\" to match.\n\nor something like that?\n\n> @@ -468,6 +469,7 @@ extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);\n> extern int get_common_dir(struct strbuf *sb, const char *gitdir);\n> extern const char *get_git_namespace(void);\n> extern const char *strip_namespace(const char *namespaced_ref);\n> +extern const char *get_super_prefix();\n\nNice, but \n\n extern const char *get_super_prefix(void);\n\n> +const char *get_super_prefix()\n\nLikewise.\n\n> +{\n> +\tif (!super_prefix)\n> +\t\tsuper_prefix = getenv(GIT_SUPER_PREFIX_ENVIRONMENT);\n> +\treturn super_prefix;\n> +}\n\nGood.\n\n> \tcommit_pager_choice();\n> \n> +\tif (!help && get_super_prefix()) {\n> +\t\tif (!(p->option & SUPPORT_SUPER_PREFIX))\n> +\t\t\tdie(\"%s doesn't support --super-prefix\", p->cmd);\n> +\t\tif (prefix)\n> +\t\t\tdie(\"can't have both a prefix and super-prefix\");\n> +\t}\n\nNice. I'd phrase the latter as \"can't use--super-prefix from a\nsubdirectory\", though.\n\n"},{"id":"302851","messageId":"xmqqvaxfhdft.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1475099443-145608-3-git-send-email-bmwill@google.com","subject":"Re: [PATCH v5 2/4] ls-files: optionally recurse into submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-28T22:22:46Z","receivedAt":"2016-09-28T22:23:03Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> +--recurse-submodules::\n> +\tRecursively calls ls-files on each submodule in the repository.\n> +\tCurrently there is only support for the --cached mode.\n\nGood to describe it, but at this step, there is only support for the\n\"--cached\" mode and it does not take a pathspec.\n\nAlso as the series progresses, I would expect this to be updated in\npatches $n/4 (2 < $n).\n\n> +\tif (recurse_submodules && argc)\n> +\t\tdie(\"ls-files --recurse-submodules does not support path \"\n> +\t\t \"arguments\");\n\nHmm, s/path arguments/pathspec/, perhaps, as the latter is specified\nin the glossary?\n\n"},{"id":"302913","messageId":"20160929183940.vgac7by74gmglaf2@sigill.intra.peff.net","threadId":"44147","inReplyTo":"1475099443-145608-2-git-send-email-bmwill@google.com","subject":"Re: [PATCH v5 1/4] git: make super-prefix option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-09-29T18:39:40Z","receivedAt":"2016-09-29T18:40:03Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Wed, Sep 28, 2016 at 02:50:40PM -0700, Brandon Williams wrote:\n\n> Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX'\n> which can be used to specify a path from above a repository down to its\n> root. The immediate use of this option is by commands which have a\n> --recurse-submodule option in order to give context to submodules about\n> how they were invoked. This option is currently only allowed for\n> builtins which support a super-prefix.\n\nWhat about non-builtins?\n\nE.g., what should\n\n git --super-prefix=foo bar\n\ndo? Should the externals and scripts check the presence of\nGIT_INTERNAL_SUPER_PREFIX and barf if it is set? Most scripts would\nprobably notice eventually when calling some other builtin that doesn't\nsupport SUPER_PREFIX, but it seems hacky to count on that.\n\nThere's also the question of 3rd-party programs. If we want to be\nconservative, I think you'd want to just always bail in\nexecv_dashed_external() if --super-prefix is in use. That doesn't give\nan option for scripts to say \"hey, I support this\", but we can perhaps\nworry about loosening later.\n\n-Peff\n"},{"id":"302914","messageId":"20160929184405.GA5500@google.com","threadId":"44147","inReplyTo":"20160929183940.vgac7by74gmglaf2@sigill.intra.peff.net","subject":"Re: [PATCH v5 1/4] git: make super-prefix option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-29T18:44:05Z","receivedAt":"2016-09-29T18:44:13Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/29, Jeff King wrote:\n> On Wed, Sep 28, 2016 at 02:50:40PM -0700, Brandon Williams wrote:\n> \n> > Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX'\n> > which can be used to specify a path from above a repository down to its\n> > root. The immediate use of this option is by commands which have a\n> > --recurse-submodule option in order to give context to submodules about\n> > how they were invoked. This option is currently only allowed for\n> > builtins which support a super-prefix.\n> \n> What about non-builtins?\n> \n> E.g., what should\n> \n> git --super-prefix=foo bar\n> \n> do? Should the externals and scripts check the presence of\n> GIT_INTERNAL_SUPER_PREFIX and barf if it is set? Most scripts would\n> probably notice eventually when calling some other builtin that doesn't\n> support SUPER_PREFIX, but it seems hacky to count on that.\n> \n> There's also the question of 3rd-party programs. If we want to be\n> conservative, I think you'd want to just always bail in\n> execv_dashed_external() if --super-prefix is in use. That doesn't give\n> an option for scripts to say \"hey, I support this\", but we can perhaps\n> worry about loosening later.\n> \n> -Peff\n\nThat makes sense.\n\n-- \nBrandon Williams\n"},{"id":"302952","messageId":"1475185723-36871-1-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475099443-145608-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 0/4] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-29T21:48:39Z","receivedAt":"2016-09-29T21:50:19Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Minor fixes per the comments on version 5.\n\nBrandon Williams (4):\n git: make super-prefix option\n ls-files: optionally recurse into submodules\n ls-files: pass through safe options for --recurse-submodules\n ls-files: add pathspec matching for submodules\n\n Documentation/git-ls-files.txt | 7 +-\n Documentation/git.txt | 6 +\n builtin/ls-files.c | 202 ++++++++++++++++++++++++-------\n cache.h | 2 +\n dir.c | 46 +++++++-\n dir.h | 4 +\n environment.c | 10 ++\n git.c | 28 ++++-\n t/t3007-ls-files-recurse-submodules.sh | 209 +++++++++++++++++++++++++++++++++\n 9 files changed, 470 insertions(+), 44 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\n-- \n2.10.0\n\n"},{"id":"302953","messageId":"1475185723-36871-2-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475185723-36871-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 1/4] git: make super-prefix option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-29T21:48:40Z","receivedAt":"2016-09-29T21:50:22Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX'\nwhich can be used to specify a path from above a repository down to its\nroot. When such a super-prefix is specified, the paths reported by Git\nare prefixed with it to make them relative to that directory \"above\".\nThe paths given by the user on the command line\n(e.g. \"git subcmd --output-file=path/to/a/file\" and pathspecs) are taken\nrelative to the directory \"above\" to match.\n\nThe immediate use of this option is by commands which have a\n--recurse-submodule option in order to give context to submodules about\nhow they were invoked. This option is currently only allowed for\nbuiltins which support a super-prefix.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git.txt | 6 ++++++\n cache.h | 2 ++\n environment.c | 10 ++++++++++\n git.c | 26 ++++++++++++++++++++++++++\n 4 files changed, 44 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 7913fc2..2188ae6 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -13,6 +13,7 @@ SYNOPSIS\n [--exec-path[=]] [--html-path] [--man-path] [--info-path]\n [-p|--paginate|--no-pager] [--no-replace-objects] [--bare]\n [--git-dir=] [--work-tree=] [--namespace=]\n+ [--super-prefix=]\n []\n \n DESCRIPTION\n@@ -601,6 +602,11 @@ foo.bar= ...`) sets `foo.bar` to the empty string.\n \tdetails. Equivalent to setting the `GIT_NAMESPACE` environment\n \tvariable.\n \n+--super-prefix=::\n+\tCurrently for internal use only. Set a prefix which gives a path from\n+\tabove a repository down to its root. One use is to give submodules\n+\tcontext about the superproject that invoked it.\n+\n --bare::\n \tTreat the repository as a bare repository. If GIT_DIR\n \tenvironment is not set, it is set to the current working\ndiff --git a/cache.h b/cache.h\nindex 3556326..8cf495d 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)\n #define GIT_NAMESPACE_ENVIRONMENT \"GIT_NAMESPACE\"\n #define GIT_WORK_TREE_ENVIRONMENT \"GIT_WORK_TREE\"\n #define GIT_PREFIX_ENVIRONMENT \"GIT_PREFIX\"\n+#define GIT_SUPER_PREFIX_ENVIRONMENT \"GIT_INTERNAL_SUPER_PREFIX\"\n #define DEFAULT_GIT_DIR_ENVIRONMENT \".git\"\n #define DB_ENVIRONMENT \"GIT_OBJECT_DIRECTORY\"\n #define INDEX_ENVIRONMENT \"GIT_INDEX_FILE\"\n@@ -468,6 +469,7 @@ extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);\n extern int get_common_dir(struct strbuf *sb, const char *gitdir);\n extern const char *get_git_namespace(void);\n extern const char *strip_namespace(const char *namespaced_ref);\n+extern const char *get_super_prefix(void);\n extern const char *get_git_work_tree(void);\n \n /*\ndiff --git a/environment.c b/environment.c\nindex ca72464..13f3d70 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -100,6 +100,8 @@ static char *work_tree;\n static const char *namespace;\n static size_t namespace_len;\n \n+static const char *super_prefix;\n+\n static const char *git_dir, *git_common_dir;\n static char *git_object_dir, *git_index_file, *git_graft_file;\n int git_db_env, git_index_env, git_graft_env, git_common_dir_env;\n@@ -120,6 +122,7 @@ const char * const local_repo_env[] = {\n \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n \tGIT_PREFIX_ENVIRONMENT,\n+\tGIT_SUPER_PREFIX_ENVIRONMENT,\n \tGIT_SHALLOW_FILE_ENVIRONMENT,\n \tGIT_COMMON_DIR_ENVIRONMENT,\n \tNULL\n@@ -222,6 +225,13 @@ const char *strip_namespace(const char *namespaced_ref)\n \treturn namespaced_ref + namespace_len;\n }\n \n+const char *get_super_prefix(void)\n+{\n+\tif (!super_prefix)\n+\t\tsuper_prefix = getenv(GIT_SUPER_PREFIX_ENVIRONMENT);\n+\treturn super_prefix;\n+}\n+\n static int git_work_tree_initialized;\n \n /*\ndiff --git a/git.c b/git.c\nindex 1c61151..f756b62 100644\n--- a/git.c\n+++ b/git.c\n@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n \t\t\tsetenv(GIT_WORK_TREE_ENVIRONMENT, cmd, 1);\n \t\t\tif (envchanged)\n \t\t\t\t*envchanged = 1;\n+\t\t} else if (!strcmp(cmd, \"--super-prefix\")) {\n+\t\t\tif (*argc < 2) {\n+\t\t\t\tfprintf(stderr, \"No prefix given for --super-prefix.\\n\" );\n+\t\t\t\tusage(git_usage_string);\n+\t\t\t}\n+\t\t\tsetenv(GIT_SUPER_PREFIX_ENVIRONMENT, (*argv)[1], 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n+\t\t\t(*argv)++;\n+\t\t\t(*argc)--;\n+\t\t} else if (skip_prefix(cmd, \"--super-prefix=\", &cmd)) {\n+\t\t\tsetenv(GIT_SUPER_PREFIX_ENVIRONMENT, cmd, 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--bare\")) {\n \t\t\tchar *cwd = xgetcwd();\n \t\t\tis_bare_repository_cfg = 1;\n@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)\n * RUN_SETUP for reading from the configuration file.\n */\n #define NEED_WORK_TREE\t\t(1<<3)\n+#define SUPPORT_SUPER_PREFIX\t(1<<4)\n \n struct cmd_struct {\n \tconst char *cmd;\n@@ -344,6 +359,13 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n \t}\n \tcommit_pager_choice();\n \n+\tif (!help && get_super_prefix()) {\n+\t\tif (!(p->option & SUPPORT_SUPER_PREFIX))\n+\t\t\tdie(\"%s doesn't support --super-prefix\", p->cmd);\n+\t\tif (prefix)\n+\t\t\tdie(\"can't use --super-prefix from a subdirectory\");\n+\t}\n+\n \tif (!help && p->option & NEED_WORK_TREE)\n \t\tsetup_work_tree();\n \n@@ -558,6 +580,10 @@ static void execv_dashed_external(const char **argv)\n \tconst char *tmp;\n \tint status;\n \n+\tif (get_super_prefix()) {\n+\t\tdie(\"%s doesn't support --super-prefix\", argv[0]);\n+\t}\n+\n \tif (use_pager == -1)\n \t\tuse_pager = check_pager_config(argv[0]);\n \tcommit_pager_choice();\n-- \n2.10.0\n\n"},{"id":"302954","messageId":"1475185723-36871-3-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475185723-36871-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 2/4] ls-files: optionally recurse into submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-29T21:48:41Z","receivedAt":"2016-09-29T21:50:36Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Allow ls-files to recognize submodules in order to retrieve a list of\nfiles from a repository's submodules. This is done by forking off a\nprocess to recursively call ls-files on all submodules. Use top-level\n--super-prefix option to pass a path to the submodule which it can\nuse to prepend to output or pathspec matching logic.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git-ls-files.txt | 8 +-\n builtin/ls-files.c | 138 ++++++++++++++++++++++++---------\n git.c | 2 +-\n t/t3007-ls-files-recurse-submodules.sh | 100 ++++++++++++++++++++++++\n 4 files changed, 208 insertions(+), 40 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\ndiff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\nindex 0d933ac..ea01d45 100644\n--- a/Documentation/git-ls-files.txt\n+++ b/Documentation/git-ls-files.txt\n@@ -18,7 +18,8 @@ SYNOPSIS\n \t\t[--exclude-per-directory=]\n \t\t[--exclude-standard]\n \t\t[--error-unmatch] [--with-tree=]\n-\t\t[--full-name] [--abbrev] [--] [...]\n+\t\t[--full-name] [--recurse-submodules]\n+\t\t[--abbrev] [--] [...]\n \n DESCRIPTION\n -----------\n@@ -137,6 +138,11 @@ a space) at the start of each line:\n \toption forces paths to be output relative to the project\n \ttop directory.\n \n+--recurse-submodules::\n+\tRecursively calls ls-files on each submodule in the repository.\n+\tCurrently there is only support for the --cached mode without a\n+\tpathspec.\n+\n --abbrev[=]::\n \tInstead of showing the full 40-byte hexadecimal object\n \tlines, show only a partial prefix.\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 00ea91a..63befed 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -14,6 +14,7 @@\n #include \"resolve-undo.h\"\n #include \"string-list.h\"\n #include \"pathspec.h\"\n+#include \"run-command.h\"\n \n static int abbrev;\n static int show_deleted;\n@@ -28,8 +29,10 @@ static int show_valid_bit;\n static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n+static int recurse_submodules;\n \n static const char *prefix;\n+static const char *super_prefix;\n static int max_prefix_len;\n static int prefix_len;\n static struct pathspec pathspec;\n@@ -68,11 +71,24 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)\n static void write_name(const char *name)\n {\n \t/*\n+\t * Prepend the super_prefix to name to construct the full_name to be\n+\t * written.\n+\t */\n+\tstruct strbuf full_name = STRBUF_INIT;\n+\tif (super_prefix) {\n+\t\tstrbuf_addstr(&full_name, super_prefix);\n+\t\tstrbuf_addstr(&full_name, name);\n+\t\tname = full_name.buf;\n+\t}\n+\n+\t/*\n \t * With \"--full-name\", prefix_len=0; this caller needs to pass\n \t * an empty string in that case (a NULL is good for \"\").\n \t */\n \twrite_name_quoted_relative(name, prefix_len ? prefix : NULL,\n \t\t\t\t stdout, line_terminator);\n+\n+\tstrbuf_release(&full_name);\n }\n \n static void show_dir_entry(const char *tag, struct dir_entry *ent)\n@@ -152,55 +168,84 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/**\n+ * Recursively call ls-files on a submodule\n+ */\n+static void show_gitlink(const struct cache_entry *ce)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tint status;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n+\t\t\t super_prefix ? super_prefix : \"\",\n+\t\t\t ce->name);\n+\targv_array_push(&cp.args, \"ls-files\");\n+\targv_array_push(&cp.args, \"--recurse-submodules\");\n+\n+\tcp.git_cmd = 1;\n+\tcp.dir = ce->name;\n+\tstatus = run_command(&cp);\n+\tif (status)\n+\t\texit(status);\n+}\n+\n static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n {\n+\tstruct strbuf name = STRBUF_INIT;\n \tint len = max_prefix_len;\n+\tif (super_prefix)\n+\t\tstrbuf_addstr(&name, super_prefix);\n+\tstrbuf_addstr(&name, ce->name);\n \n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (!match_pathspec(&pathspec, ce->name, ce_namelen(ce),\n-\t\t\t len, ps_matched,\n-\t\t\t S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)))\n-\t\treturn;\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\t\tshow_gitlink(ce);\n+\t} else if (match_pathspec(&pathspec, name.buf, name.len,\n+\t\t\t\t len, ps_matched,\n+\t\t\t\t S_ISDIR(ce->ce_mode) ||\n+\t\t\t\t S_ISGITLINK(ce->ce_mode))) {\n+\t\tif (tag && *tag && show_valid_bit &&\n+\t\t (ce->ce_flags & CE_VALID)) {\n+\t\t\tstatic char alttag[4];\n+\t\t\tmemcpy(alttag, tag, 3);\n+\t\t\tif (isalpha(tag[0]))\n+\t\t\t\talttag[0] = tolower(tag[0]);\n+\t\t\telse if (tag[0] == '?')\n+\t\t\t\talttag[0] = '!';\n+\t\t\telse {\n+\t\t\t\talttag[0] = 'v';\n+\t\t\t\talttag[1] = tag[0];\n+\t\t\t\talttag[2] = ' ';\n+\t\t\t\talttag[3] = 0;\n+\t\t\t}\n+\t\t\ttag = alttag;\n+\t\t}\n \n-\tif (tag && *tag && show_valid_bit &&\n-\t (ce->ce_flags & CE_VALID)) {\n-\t\tstatic char alttag[4];\n-\t\tmemcpy(alttag, tag, 3);\n-\t\tif (isalpha(tag[0]))\n-\t\t\talttag[0] = tolower(tag[0]);\n-\t\telse if (tag[0] == '?')\n-\t\t\talttag[0] = '!';\n-\t\telse {\n-\t\t\talttag[0] = 'v';\n-\t\t\talttag[1] = tag[0];\n-\t\t\talttag[2] = ' ';\n-\t\t\talttag[3] = 0;\n+\t\tif (!show_stage) {\n+\t\t\tfputs(tag, stdout);\n+\t\t} else {\n+\t\t\tprintf(\"%s%06o %s %d\\t\",\n+\t\t\t tag,\n+\t\t\t ce->ce_mode,\n+\t\t\t find_unique_abbrev(ce->sha1,abbrev),\n+\t\t\t ce_stage(ce));\n+\t\t}\n+\t\twrite_eolinfo(ce, ce->name);\n+\t\twrite_name(ce->name);\n+\t\tif (debug_mode) {\n+\t\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n+\n+\t\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n+\t\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n+\t\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n+\t\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n+\t\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n \t\t}\n-\t\ttag = alttag;\n \t}\n \n-\tif (!show_stage) {\n-\t\tfputs(tag, stdout);\n-\t} else {\n-\t\tprintf(\"%s%06o %s %d\\t\",\n-\t\t tag,\n-\t\t ce->ce_mode,\n-\t\t find_unique_abbrev(ce->sha1,abbrev),\n-\t\t ce_stage(ce));\n-\t}\n-\twrite_eolinfo(ce, ce->name);\n-\twrite_name(ce->name);\n-\tif (debug_mode) {\n-\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n-\n-\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n-\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n-\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n-\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n-\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n-\t}\n+\tstrbuf_release(&name);\n }\n \n static void show_ru_info(void)\n@@ -468,6 +513,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\t{ OPTION_SET_INT, 0, \"full-name\", &prefix_len, NULL,\n \t\t\tN_(\"make the output relative to the project top directory\"),\n \t\t\tPARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL },\n+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\tN_(\"recurse through submodules\")),\n \t\tOPT_BOOL(0, \"error-unmatch\", &error_unmatch,\n \t\t\tN_(\"if any is not in the index, treat this as an error\")),\n \t\tOPT_STRING(0, \"with-tree\", &with_tree, N_(\"tree-ish\"),\n@@ -484,6 +531,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tprefix = cmd_prefix;\n \tif (prefix)\n \t\tprefix_len = strlen(prefix);\n+\tsuper_prefix = get_super_prefix();\n \tgit_config(git_default_config, NULL);\n \n \tif (read_cache() < 0)\n@@ -519,6 +567,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules &&\n+\t (show_stage || show_deleted || show_others || show_unmerged ||\n+\t show_killed || show_modified || show_resolve_undo ||\n+\t show_valid_bit || show_tag || show_eol || with_tree ||\n+\t (line_terminator == '\\0')))\n+\t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n+\n+\tif (recurse_submodules && error_unmatch)\n+\t\tdie(\"ls-files --recurse-submodules does not support \"\n+\t\t \"--error-unmatch\");\n+\n+\tif (recurse_submodules && argc)\n+\t\tdie(\"ls-files --recurse-submodules does not support pathspec\");\n+\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\ndiff --git a/git.c b/git.c\nindex f756b62..1bbd5ae 100644\n--- a/git.c\n+++ b/git.c\n@@ -443,7 +443,7 @@ static struct cmd_struct commands[] = {\n \t{ \"init-db\", cmd_init_db },\n \t{ \"interpret-trailers\", cmd_interpret_trailers, RUN_SETUP_GENTLY },\n \t{ \"log\", cmd_log, RUN_SETUP },\n-\t{ \"ls-files\", cmd_ls_files, RUN_SETUP },\n+\t{ \"ls-files\", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },\n \t{ \"ls-remote\", cmd_ls_remote, RUN_SETUP_GENTLY },\n \t{ \"ls-tree\", cmd_ls_tree, RUN_SETUP },\n \t{ \"mailinfo\", cmd_mailinfo },\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nnew file mode 100755\nindex 0000000..b5a53c3\n--- /dev/null\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -0,0 +1,100 @@\n+#!/bin/sh\n+\n+test_description='Test ls-files recurse-submodules feature\n+\n+This test verifies the recurse-submodules feature correctly lists files from\n+submodules.\n+'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup directory structure and submodules' '\n+\techo a >a &&\n+\tmkdir b &&\n+\techo b >b/b &&\n+\tgit add a b &&\n+\tgit commit -m \"add a and b\" &&\n+\tgit init submodule &&\n+\techo c >submodule/c &&\n+\tgit -C submodule add c &&\n+\tgit -C submodule commit -m \"add c\" &&\n+\tgit submodule add ./submodule &&\n+\tgit commit -m \"added submodule\"\n+'\n+\n+test_expect_success 'ls-files correctly outputs files in submodule' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files does not output files not added to a repo' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\techo a >not_added &&\n+\techo b >b/not_added &&\n+\techo c >submodule/not_added &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files recurses more than 1 level' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/subsub/d\n+\tEOF\n+\n+\tgit init submodule/subsub &&\n+\techo d >submodule/subsub/d &&\n+\tgit -C submodule/subsub add d &&\n+\tgit -C submodule/subsub commit -m \"add d\" &&\n+\tgit -C submodule submodule add ./subsub &&\n+\tgit -C submodule commit -m \"added subsub\" &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support using path arguments' '\n+\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n+\ttest_i18ngrep \"does not support pathspec\" actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support --error-unmatch' '\n+\ttest_must_fail git ls-files --recurse-submodules --error-unmatch 2>actual &&\n+\ttest_i18ngrep \"does not support --error-unmatch\" actual\n+'\n+\n+test_incompatible_with_recurse_submodules () {\n+\ttest_expect_success \"--recurse-submodules and $1 are incompatible\" \"\n+\t\ttest_must_fail git ls-files --recurse-submodules $1 2>actual &&\n+\t\ttest_i18ngrep 'unsupported mode' actual\n+\t\"\n+}\n+\n+test_incompatible_with_recurse_submodules -z\n+test_incompatible_with_recurse_submodules -v\n+test_incompatible_with_recurse_submodules -t\n+test_incompatible_with_recurse_submodules --deleted\n+test_incompatible_with_recurse_submodules --modified\n+test_incompatible_with_recurse_submodules --others\n+test_incompatible_with_recurse_submodules --stage\n+test_incompatible_with_recurse_submodules --killed\n+test_incompatible_with_recurse_submodules --unmerged\n+test_incompatible_with_recurse_submodules --eol\n+\n+test_done\n-- \n2.10.0\n\n"},{"id":"302955","messageId":"1475185723-36871-4-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475185723-36871-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 3/4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-29T21:48:42Z","receivedAt":"2016-09-29T21:50:50Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pass through some known-safe options when recursing into submodules.\n(--cached, --stage, -v, -t, -z, --debug, --eol)\n\nOther options are compiled into an argv_array but if an unsafe option is\ngiven the caller will be errored out.\n\nSigned-off-by: Brandon Williams \n---\n builtin/ls-files.c | 51 ++++++++++++++++++++++++++++++++--\n t/t3007-ls-files-recurse-submodules.sh | 17 ++++++++----\n 2 files changed, 60 insertions(+), 8 deletions(-)\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 63befed..6f744ef 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -30,6 +30,7 @@ static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n static int recurse_submodules;\n+static struct argv_array submodules_options = ARGV_ARRAY_INIT;\n \n static const char *prefix;\n static const char *super_prefix;\n@@ -168,6 +169,45 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/*\n+ * Compile an argv_array with all of the options supported by --recurse_submodules\n+ */\n+static void compile_submodule_options(const struct dir_struct *dir, int show_tag)\n+{\n+\tif (line_terminator == '\\0')\n+\t\targv_array_push(&submodules_options, \"-z\");\n+\tif (show_tag)\n+\t\targv_array_push(&submodules_options, \"-t\");\n+\tif (show_valid_bit)\n+\t\targv_array_push(&submodules_options, \"-v\");\n+\tif (show_cached)\n+\t\targv_array_push(&submodules_options, \"--cached\");\n+\tif (show_deleted)\n+\t\targv_array_push(&submodules_options, \"--deleted\");\n+\tif (show_modified)\n+\t\targv_array_push(&submodules_options, \"--modified\");\n+\tif (show_others)\n+\t\targv_array_push(&submodules_options, \"--others\");\n+\tif (dir->flags & DIR_SHOW_IGNORED)\n+\t\targv_array_push(&submodules_options, \"--ignored\");\n+\tif (show_stage)\n+\t\targv_array_push(&submodules_options, \"--stage\");\n+\tif (show_killed)\n+\t\targv_array_push(&submodules_options, \"--killed\");\n+\tif (dir->flags & DIR_SHOW_OTHER_DIRECTORIES)\n+\t\targv_array_push(&submodules_options, \"--directory\");\n+\tif (!(dir->flags & DIR_SHOW_OTHER_DIRECTORIES))\n+\t\targv_array_push(&submodules_options, \"--empty-directory\");\n+\tif (show_unmerged)\n+\t\targv_array_push(&submodules_options, \"--unmerged\");\n+\tif (show_resolve_undo)\n+\t\targv_array_push(&submodules_options, \"--resolve-undo\");\n+\tif (show_eol)\n+\t\targv_array_push(&submodules_options, \"--eol\");\n+\tif (debug_mode)\n+\t\targv_array_push(&submodules_options, \"--debug\");\n+}\n+\n /**\n * Recursively call ls-files on a submodule\n */\n@@ -182,6 +222,9 @@ static void show_gitlink(const struct cache_entry *ce)\n \targv_array_push(&cp.args, \"ls-files\");\n \targv_array_push(&cp.args, \"--recurse-submodules\");\n \n+\t/* add supported options */\n+\targv_array_pushv(&cp.args, submodules_options.argv);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -567,11 +610,13 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules)\n+\t\tcompile_submodule_options(&dir, show_tag);\n+\n \tif (recurse_submodules &&\n-\t (show_stage || show_deleted || show_others || show_unmerged ||\n+\t (show_deleted || show_others || show_unmerged ||\n \t show_killed || show_modified || show_resolve_undo ||\n-\t show_valid_bit || show_tag || show_eol || with_tree ||\n-\t (line_terminator == '\\0')))\n+\t with_tree))\n \t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n \n \tif (recurse_submodules && error_unmatch)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex b5a53c3..e76fa30 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'ls-files correctly outputs files in submodule with -z' '\n+\tlf_to_nul >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules -z >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'ls-files does not output files not added to a repo' '\n \tcat >expect <<-\\EOF &&\n \t.gitmodules\n@@ -86,15 +98,10 @@ test_incompatible_with_recurse_submodules () {\n \t\"\n }\n \n-test_incompatible_with_recurse_submodules -z\n-test_incompatible_with_recurse_submodules -v\n-test_incompatible_with_recurse_submodules -t\n test_incompatible_with_recurse_submodules --deleted\n test_incompatible_with_recurse_submodules --modified\n test_incompatible_with_recurse_submodules --others\n-test_incompatible_with_recurse_submodules --stage\n test_incompatible_with_recurse_submodules --killed\n test_incompatible_with_recurse_submodules --unmerged\n-test_incompatible_with_recurse_submodules --eol\n \n test_done\n-- \n2.10.0\n\n"},{"id":"302956","messageId":"1475185723-36871-5-git-send-email-bmwill@google.com","threadId":"44147","inReplyTo":"1475185723-36871-1-git-send-email-bmwill@google.com","subject":"[PATCH v6 4/4] ls-files: add pathspec matching for submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-29T21:48:43Z","receivedAt":"2016-09-29T21:50:53Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pathspecs can be a bit tricky when trying to apply them to submodules.\nThe main challenge is that the pathspecs will be with respect to the\nsuperproject and not with respect to paths in the submodule. The\napproach this patch takes is to pass in the identical pathspec from the\nsuperproject to the submodule in addition to the submodule-prefix, which\nis the path from the root of the superproject to the submodule, and then\nwe can compare an entry in the submodule prepended with the\nsubmodule-prefix to the pathspec in order to determine if there is a\nmatch.\n\nThis patch also permits the pathspec logic to perform a prefix match against\nsubmodules since a pathspec could refer to a file inside of a submodule.\nDue to limitations in the wildmatch logic, a prefix match is only done\nliterally. If any wildcard character is encountered we'll simply punt\nand produce a false positive match. More accurate matching will be done\nonce inside the submodule. This is due to the superproject not knowing\nwhat files could exist in the submodule.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git-ls-files.txt | 3 +-\n builtin/ls-files.c | 27 +++++++--\n dir.c | 46 +++++++++++++-\n dir.h | 4 ++\n t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++-\n 5 files changed, 175 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\nindex ea01d45..51ec9a1 100644\n--- a/Documentation/git-ls-files.txt\n+++ b/Documentation/git-ls-files.txt\n@@ -140,8 +140,7 @@ a space) at the start of each line:\n \n --recurse-submodules::\n \tRecursively calls ls-files on each submodule in the repository.\n-\tCurrently there is only support for the --cached mode without a\n-\tpathspec.\n+\tCurrently there is only support for the --cached.\n \n --abbrev[=]::\n \tInstead of showing the full 40-byte hexadecimal object\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 6f744ef..82ec811 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -215,6 +215,7 @@ static void show_gitlink(const struct cache_entry *ce)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tint status;\n+\tint i;\n \n \targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n \t\t\t super_prefix ? super_prefix : \"\",\n@@ -225,6 +226,15 @@ static void show_gitlink(const struct cache_entry *ce)\n \t/* add supported options */\n \targv_array_pushv(&cp.args, submodules_options.argv);\n \n+\t/*\n+\t * Pass in the original pathspec args. The submodule will be\n+\t * responsible for prepending the 'submodule_prefix' prior to comparing\n+\t * against the pathspec for matches.\n+\t */\n+\targv_array_push(&cp.args, \"--\");\n+\tfor (i = 0; i < pathspec.nr; i++)\n+\t\targv_array_push(&cp.args, pathspec.items[i].original);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -243,7 +253,8 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&\n+\t submodule_path_match(&pathspec, name.buf, ps_matched)) {\n \t\tshow_gitlink(ce);\n \t} else if (match_pathspec(&pathspec, name.buf, name.len,\n \t\t\t\t len, ps_matched,\n@@ -623,16 +634,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\tdie(\"ls-files --recurse-submodules does not support \"\n \t\t \"--error-unmatch\");\n \n-\tif (recurse_submodules && argc)\n-\t\tdie(\"ls-files --recurse-submodules does not support pathspec\");\n-\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\n \t\t prefix, argv);\n \n-\t/* Find common prefix for all pathspec's */\n-\tmax_prefix = common_prefix(&pathspec);\n+\t/*\n+\t * Find common prefix for all pathspec's\n+\t * This is used as a performance optimization which unfortunately cannot\n+\t * be done when recursing into submodules\n+\t */\n+\tif (recurse_submodules)\n+\t\tmax_prefix = NULL;\n+\telse\n+\t\tmax_prefix = common_prefix(&pathspec);\n \tmax_prefix_len = max_prefix ? strlen(max_prefix) : 0;\n \n \t/* Treat unmatching pathspec elements as errors */\ndiff --git a/dir.c b/dir.c\nindex 0ea235f..28e9736 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -207,8 +207,9 @@ int within_depth(const char *name, int namelen,\n \treturn 1;\n }\n \n-#define DO_MATCH_EXCLUDE 1\n-#define DO_MATCH_DIRECTORY 2\n+#define DO_MATCH_EXCLUDE (1<<0)\n+#define DO_MATCH_DIRECTORY (1<<1)\n+#define DO_MATCH_SUBMODULE (1<<2)\n \n /*\n * Does 'match' match the given name?\n@@ -283,6 +284,32 @@ static int match_pathspec_item(const struct pathspec_item *item, int prefix,\n \t\t\t item->nowildcard_len - prefix))\n \t\treturn MATCHED_FNMATCH;\n \n+\t/* Perform checks to see if \"name\" is a super set of the pathspec */\n+\tif (flags & DO_MATCH_SUBMODULE) {\n+\t\t/* name is a literal prefix of the pathspec */\n+\t\tif ((namelen < matchlen) &&\n+\t\t (match[namelen] == '/') &&\n+\t\t !ps_strncmp(item, match, name, namelen))\n+\t\t\treturn MATCHED_RECURSIVELY;\n+\n+\t\t/* name\" doesn't match up to the first wild character */\n+\t\tif (item->nowildcard_len < item->len &&\n+\t\t ps_strncmp(item, match, name,\n+\t\t\t item->nowildcard_len - prefix))\n+\t\t\treturn 0;\n+\n+\t\t/*\n+\t\t * Here is where we would perform a wildmatch to check if\n+\t\t * \"name\" can be matched as a directory (or a prefix) against\n+\t\t * the pathspec. Since wildmatch doesn't have this capability\n+\t\t * at the present we have to punt and say that it is a match,\n+\t\t * potentially returning a false positive\n+\t\t * The submodules themselves will be able to perform more\n+\t\t * accurate matching to determine if the pathspec matches.\n+\t\t */\n+\t\treturn MATCHED_RECURSIVELY;\n+\t}\n+\n \treturn 0;\n }\n \n@@ -386,6 +413,21 @@ int match_pathspec(const struct pathspec *ps,\n \treturn negative ? 0 : positive;\n }\n \n+/**\n+ * Check if a submodule is a superset of the pathspec\n+ */\n+int submodule_path_match(const struct pathspec *ps,\n+\t\t\t const char *submodule_name,\n+\t\t\t char *seen)\n+{\n+\tint matched = do_match_pathspec(ps, submodule_name,\n+\t\t\t\t\tstrlen(submodule_name),\n+\t\t\t\t\t0, seen,\n+\t\t\t\t\tDO_MATCH_DIRECTORY |\n+\t\t\t\t\tDO_MATCH_SUBMODULE);\n+\treturn matched;\n+}\n+\n int report_path_error(const char *ps_matched,\n \t\t const struct pathspec *pathspec,\n \t\t const char *prefix)\ndiff --git a/dir.h b/dir.h\nindex da1a858..97c83bb 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item,\n \t\t const char *pattern, const char *string,\n \t\t int prefix);\n \n+extern int submodule_path_match(const struct pathspec *ps,\n+\t\t\t\tconst char *submodule_name,\n+\t\t\t\tchar *seen);\n+\n static inline int ce_path_match(const struct cache_entry *ce,\n \t\t\t\tconst struct pathspec *pathspec,\n \t\t\t\tchar *seen)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex e76fa30..5475855 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -81,9 +81,111 @@ test_expect_success 'ls-files recurses more than 1 level' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success '--recurse-submodules does not support using path arguments' '\n-\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n-\ttest_i18ngrep \"does not support pathspec\" actual\n+test_expect_success '--recurse-submodules and pathspecs setup' '\n+\techo e >submodule/subsub/e.txt &&\n+\tgit -C submodule/subsub add e.txt &&\n+\tgit -C submodule/subsub commit -m \"adding e.txt\" &&\n+\techo f >submodule/f.TXT &&\n+\techo g >submodule/g.txt &&\n+\tgit -C submodule add f.TXT g.txt &&\n+\tgit -C submodule commit -m \"add f and g\" &&\n+\techo h >h.txt &&\n+\tmkdir sib &&\n+\techo sib >sib/file &&\n+\tgit add h.txt sib/file &&\n+\tgit commit -m \"add h and sib/file\" &&\n+\tgit init sub &&\n+\techo sub >sub/file &&\n+\tgit -C sub add file &&\n+\tgit -C sub commit -m \"add file\" &&\n+\tgit submodule add ./sub &&\n+\tgit commit -m \"added sub\" &&\n+\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\th.txt\n+\tsib/file\n+\tsub/file\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/d\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual &&\n+\tcat actual &&\n+\tgit ls-files --recurse-submodules \"*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" \":(exclude)submodule/subsub/*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"sub\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su*/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su?/file\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsib/file\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"s??/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s???file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s*file\" >actual &&\n+\ttest_cmp expect actual\n '\n \n test_expect_success '--recurse-submodules does not support --error-unmatch' '\n-- \n2.10.0\n\n"},{"id":"302966","messageId":"xmqqy42ab5ww.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"1475185723-36871-4-git-send-email-bmwill@google.com","subject":"Re: [PATCH v6 3/4] ls-files: pass through safe options for --recurse-submodules","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-09-30T00:14:07Z","receivedAt":"2016-09-30T00:14:20Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Brandon Williams writes:\n\n> +static void compile_submodule_options(const struct dir_struct *dir, int show_tag)\n> +{\n> +\tif (line_terminator == '\\0')\n> +\t\targv_array_push(&submodules_options, \"-z\");\n> +\tif (show_tag)\n> +\t\targv_array_push(&submodules_options, \"-t\");\n> +\tif (show_valid_bit)\n> +\t\targv_array_push(&submodules_options, \"-v\");\n> +\tif (show_cached)\n> +\t\targv_array_push(&submodules_options, \"--cached\");\n> +\tif (show_deleted)\n> +\t\targv_array_push(&submodules_options, \"--deleted\");\n> +\tif (show_modified)\n> +\t\targv_array_push(&submodules_options, \"--modified\");\n> +\tif (show_others)\n> +\t\targv_array_push(&submodules_options, \"--others\");\n> +\tif (dir->flags & DIR_SHOW_IGNORED)\n> +\t\targv_array_push(&submodules_options, \"--ignored\");\n> +\tif (show_stage)\n> +\t\targv_array_push(&submodules_options, \"--stage\");\n> +\tif (show_killed)\n> +\t\targv_array_push(&submodules_options, \"--killed\");\n> +\tif (dir->flags & DIR_SHOW_OTHER_DIRECTORIES)\n> +\t\targv_array_push(&submodules_options, \"--directory\");\n> +\tif (!(dir->flags & DIR_SHOW_OTHER_DIRECTORIES))\n> +\t\targv_array_push(&submodules_options, \"--empty-directory\");\n> +\tif (show_unmerged)\n> +\t\targv_array_push(&submodules_options, \"--unmerged\");\n> +\tif (show_resolve_undo)\n> +\t\targv_array_push(&submodules_options, \"--resolve-undo\");\n> +\tif (show_eol)\n> +\t\targv_array_push(&submodules_options, \"--eol\");\n> +\tif (debug_mode)\n> +\t\targv_array_push(&submodules_options, \"--debug\");\n> +}\n\nWith this and 4/4 applied, the documentation still says \"--cached\"\nis the only supported option.\n\nDoes it really make sense to pass all of these? I understand \"-z\"\nand I suspect things like \"-t\" and \"-v\" that affect \"how\" things are\nshown may also happen to work, but I am not sure how much it makes\nsense for options that affect \"what\" things are shown.\n\nWhat does it even mean to ask for say \"--unmerged\" to be shown, for\nexample, from the superproject? Recurse into submodules whose cache\nentries in the index of the superproject are unmerged, or something\nelse?\n\nI am inclined to say that it is probably better to keep the\n\"--cached only\" as documented, at least on the \"what are shown\"\nside.\n\nThanks.\n"},{"id":"302989","messageId":"20160930163347.GA11126@google.com","threadId":"44147","inReplyTo":"xmqqy42ab5ww.fsf@gitster.mtv.corp.google.com","subject":"Re: [PATCH v6 3/4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-30T16:33:47Z","receivedAt":"2016-09-30T16:33:56Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 09/29, Junio C Hamano wrote:\n> Brandon Williams writes:\n> \n> > +static void compile_submodule_options(const struct dir_struct *dir, int show_tag)\n> > +{\n> > +\tif (line_terminator == '\\0')\n> > +\t\targv_array_push(&submodules_options, \"-z\");\n> > +\tif (show_tag)\n> > +\t\targv_array_push(&submodules_options, \"-t\");\n> > +\tif (show_valid_bit)\n> > +\t\targv_array_push(&submodules_options, \"-v\");\n> > +\tif (show_cached)\n> > +\t\targv_array_push(&submodules_options, \"--cached\");\n> > +\tif (show_deleted)\n> > +\t\targv_array_push(&submodules_options, \"--deleted\");\n> > +\tif (show_modified)\n> > +\t\targv_array_push(&submodules_options, \"--modified\");\n> > +\tif (show_others)\n> > +\t\targv_array_push(&submodules_options, \"--others\");\n> > +\tif (dir->flags & DIR_SHOW_IGNORED)\n> > +\t\targv_array_push(&submodules_options, \"--ignored\");\n> > +\tif (show_stage)\n> > +\t\targv_array_push(&submodules_options, \"--stage\");\n> > +\tif (show_killed)\n> > +\t\targv_array_push(&submodules_options, \"--killed\");\n> > +\tif (dir->flags & DIR_SHOW_OTHER_DIRECTORIES)\n> > +\t\targv_array_push(&submodules_options, \"--directory\");\n> > +\tif (!(dir->flags & DIR_SHOW_OTHER_DIRECTORIES))\n> > +\t\targv_array_push(&submodules_options, \"--empty-directory\");\n> > +\tif (show_unmerged)\n> > +\t\targv_array_push(&submodules_options, \"--unmerged\");\n> > +\tif (show_resolve_undo)\n> > +\t\targv_array_push(&submodules_options, \"--resolve-undo\");\n> > +\tif (show_eol)\n> > +\t\targv_array_push(&submodules_options, \"--eol\");\n> > +\tif (debug_mode)\n> > +\t\targv_array_push(&submodules_options, \"--debug\");\n> > +}\n> \n> With this and 4/4 applied, the documentation still says \"--cached\"\n> is the only supported option.\n> \n> Does it really make sense to pass all of these? I understand \"-z\"\n> and I suspect things like \"-t\" and \"-v\" that affect \"how\" things are\n> shown may also happen to work, but I am not sure how much it makes\n> sense for options that affect \"what\" things are shown.\n> \n> What does it even mean to ask for say \"--unmerged\" to be shown, for\n> example, from the superproject? Recurse into submodules whose cache\n> entries in the index of the superproject are unmerged, or something\n> else?\n> \n> I am inclined to say that it is probably better to keep the\n> \"--cached only\" as documented, at least on the \"what are shown\"\n> side.\n> \n> Thanks.\n\nYou're right that probably makes the most sense for now.\n\n-- \nBrandon Williams\n"},{"id":"302990","messageId":"20160930170118.GA13951@google.com","threadId":"44147","inReplyTo":"20160930163347.GA11126@google.com","subject":"Re: [PATCH v6 3/4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-09-30T17:01:18Z","receivedAt":"2016-09-30T17:01:27Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pass through some known-safe options when recursing into submodules.\n(--cached, -v, -t, -z, --debug, --eol)\n\nIf other unsafe options are given the caller will be errored out.\n\nSigned-off-by: Brandon Williams \n---\n\nSomething more like this correct? I ditched the extra parameters and\nreworded the commit msg to reflect this.\n\n builtin/ls-files.c | 30 +++++++++++++++++++++++++++---\n t/t3007-ls-files-recurse-submodules.sh | 16 ++++++++++++----\n 2 files changed, 39 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 63befed..b6144a5 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -30,6 +30,7 @@ static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n static int recurse_submodules;\n+static struct argv_array submodules_options = ARGV_ARRAY_INIT;\n \n static const char *prefix;\n static const char *super_prefix;\n@@ -168,6 +169,25 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/*\n+ * Compile an argv_array with all of the options supported by --recurse_submodules\n+ */\n+static void compile_submodule_options(const struct dir_struct *dir, int show_tag)\n+{\n+\tif (line_terminator == '\\0')\n+\t\targv_array_push(&submodules_options, \"-z\");\n+\tif (show_tag)\n+\t\targv_array_push(&submodules_options, \"-t\");\n+\tif (show_valid_bit)\n+\t\targv_array_push(&submodules_options, \"-v\");\n+\tif (show_cached)\n+\t\targv_array_push(&submodules_options, \"--cached\");\n+\tif (show_eol)\n+\t\targv_array_push(&submodules_options, \"--eol\");\n+\tif (debug_mode)\n+\t\targv_array_push(&submodules_options, \"--debug\");\n+}\n+\n /**\n * Recursively call ls-files on a submodule\n */\n@@ -182,6 +202,9 @@ static void show_gitlink(const struct cache_entry *ce)\n \targv_array_push(&cp.args, \"ls-files\");\n \targv_array_push(&cp.args, \"--recurse-submodules\");\n \n+\t/* add supported options */\n+\targv_array_pushv(&cp.args, submodules_options.argv);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -567,11 +590,12 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules)\n+\t\tcompile_submodule_options(&dir, show_tag);\n+\n \tif (recurse_submodules &&\n \t (show_stage || show_deleted || show_others || show_unmerged ||\n-\t show_killed || show_modified || show_resolve_undo ||\n-\t show_valid_bit || show_tag || show_eol || with_tree ||\n-\t (line_terminator == '\\0')))\n+\t show_killed || show_modified || show_resolve_undo || with_tree))\n \t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n \n \tif (recurse_submodules && error_unmatch)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex b5a53c3..33a2ea7 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'ls-files correctly outputs files in submodule with -z' '\n+\tlf_to_nul >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules -z >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'ls-files does not output files not added to a repo' '\n \tcat >expect <<-\\EOF &&\n \t.gitmodules\n@@ -86,15 +98,11 @@ test_incompatible_with_recurse_submodules () {\n \t\"\n }\n \n-test_incompatible_with_recurse_submodules -z\n-test_incompatible_with_recurse_submodules -v\n-test_incompatible_with_recurse_submodules -t\n test_incompatible_with_recurse_submodules --deleted\n test_incompatible_with_recurse_submodules --modified\n test_incompatible_with_recurse_submodules --others\n test_incompatible_with_recurse_submodules --stage\n test_incompatible_with_recurse_submodules --killed\n test_incompatible_with_recurse_submodules --unmerged\n-test_incompatible_with_recurse_submodules --eol\n \n test_done\n-- \n2.10.0\n\n"},{"id":"303264","messageId":"CAGZ79kax9g-FLMhPnDBP+7wJwYT884B5bGodpopo9GKgnE6+PQ@mail.gmail.com","threadId":"44147","inReplyTo":"1475185723-36871-2-git-send-email-bmwill@google.com","subject":"Re: [PATCH v6 1/4] git: make super-prefix option","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-10-04T17:31:51Z","receivedAt":"2016-10-04T17:31:56Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Sep 29, 2016 at 2:48 PM, Brandon Williams wrote:\n\n>\n> +const char *get_super_prefix(void)\n> +{\n> + if (!super_prefix)\n> + super_prefix = getenv(GIT_SUPER_PREFIX_ENVIRONMENT);\n> + return super_prefix;\n> +}\n> +\n\nAs said earlier, is the following a valid thought:\n\n> The getenv() function returns a pointer to the value in the\n> environment, or NULL if there is no match.\n> So in case this is not set (when e.g. the user did not specify the\n> super prefix), we would probe it a couple of times.\n> The caching effect only occurs when the string is set. So this looks\n> like we save repetitive calls, but we do not always do that.\n\n>\n> + if (get_super_prefix()) {\n> + die(\"%s doesn't support --super-prefix\", argv[0]);\n> + }\n> +\n\nNit: no braces, please.\n"},{"id":"303266","messageId":"xmqqoa30uien.fsf@gitster.mtv.corp.google.com","threadId":"44147","inReplyTo":"CAGZ79kax9g-FLMhPnDBP+7wJwYT884B5bGodpopo9GKgnE6+PQ@mail.gmail.com","subject":"Re: [PATCH v6 1/4] git: make super-prefix option","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2016-10-04T17:35:12Z","receivedAt":"2016-10-04T17:35:19Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Stefan Beller writes:\n\n> On Thu, Sep 29, 2016 at 2:48 PM, Brandon Williams wrote:\n>\n>>\n>> +const char *get_super_prefix(void)\n>> +{\n>> + if (!super_prefix)\n>> + super_prefix = getenv(GIT_SUPER_PREFIX_ENVIRONMENT);\n>> + return super_prefix;\n>> +}\n>> +\n>\n> As said earlier, is the following a valid thought:\n>\n>> The getenv() function returns a pointer to the value in the\n>> environment, or NULL if there is no match.\n>> So in case this is not set (when e.g. the user did not specify the\n>> super prefix), we would probe it a couple of times.\n>> The caching effect only occurs when the string is set. So this looks\n>> like we save repetitive calls, but we do not always do that.\n\nThat reading is correct. If the code wants to do the caching, it\nshould do so correctly. Unless we expect that some callers might\nwant to be able to invalidate the cache,\n\n\tget_super_prefix(void)\n\t{\n\t\tstatic int initialized;\n\t\tif (!initialized) {\n\t\t\tsuper_prefix = getenv(...);\n\t\t\tinitialized = 1;\n\t\t}\n return super_prefix;\n\t} \n\nwould suffice.\n\nThanks for careful reading.\n"},{"id":"303267","messageId":"20161004173904.tu37rqn4rl26gabx@sigill.intra.peff.net","threadId":"44147","inReplyTo":"CAGZ79kax9g-FLMhPnDBP+7wJwYT884B5bGodpopo9GKgnE6+PQ@mail.gmail.com","subject":"Re: [PATCH v6 1/4] git: make super-prefix option","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2016-10-04T17:39:04Z","receivedAt":"2016-10-04T17:39:11Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 04, 2016 at 10:31:51AM -0700, Stefan Beller wrote:\n\n> On Thu, Sep 29, 2016 at 2:48 PM, Brandon Williams wrote:\n> \n> >\n> > +const char *get_super_prefix(void)\n> > +{\n> > + if (!super_prefix)\n> > + super_prefix = getenv(GIT_SUPER_PREFIX_ENVIRONMENT);\n> > + return super_prefix;\n> > +}\n> > +\n> \n> As said earlier, is the following a valid thought:\n> \n> > The getenv() function returns a pointer to the value in the\n> > environment, or NULL if there is no match.\n> > So in case this is not set (when e.g. the user did not specify the\n> > super prefix), we would probe it a couple of times.\n> > The caching effect only occurs when the string is set. So this looks\n> > like we save repetitive calls, but we do not always do that.\n\nI think your concern is valid. If it is not set, we will do an O(n)\nsearch through the whole environment on each call.\n\nI also think the result of getenv() needs to be copied. In some\nimplementations it persists for the life of the program, but that's not\nguaranteed; it may be overwritten by unrelated calls to getenv() or\nsetenv().\n\n-Peff\n"},{"id":"303271","messageId":"CAGZ79kbFiK0Wb6zpuyybwUOeXo-H=9oDpO8wnXihYmP8VwAHFQ@mail.gmail.com","threadId":"44147","inReplyTo":"1475185723-36871-5-git-send-email-bmwill@google.com","subject":"Re: [PATCH v6 4/4] ls-files: add pathspec matching for submodules","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-10-04T17:56:19Z","receivedAt":"2016-10-04T17:56:25Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Thu, Sep 29, 2016 at 2:48 PM, Brandon Williams wrote:\n> Pathspecs can be a bit tricky when trying to apply them to submodules.\n> The main challenge is that the pathspecs will be with respect to the\n> superproject and not with respect to paths in the submodule. The\n> approach this patch takes is to pass in the identical pathspec from the\n> superproject to the submodule in addition to the submodule-prefix, which\n> is the path from the root of the superproject to the submodule, and then\n> we can compare an entry in the submodule prepended with the\n> submodule-prefix to the pathspec in order to determine if there is a\n> match.\n>\n> This patch also permits the pathspec logic to perform a prefix match against\n> submodules since a pathspec could refer to a file inside of a submodule.\n> Due to limitations in the wildmatch logic, a prefix match is only done\n> literally. If any wildcard character is encountered we'll simply punt\n> and produce a false positive match. More accurate matching will be done\n> once inside the submodule. This is due to the superproject not knowing\n> what files could exist in the submodule.\n>\n> Signed-off-by: Brandon Williams \n> ---\n> Documentation/git-ls-files.txt | 3 +-\n> builtin/ls-files.c | 27 +++++++--\n> dir.c | 46 +++++++++++++-\n> dir.h | 4 ++\n> t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++-\n> 5 files changed, 175 insertions(+), 13 deletions(-)\n>\n> diff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\n> index ea01d45..51ec9a1 100644\n> --- a/Documentation/git-ls-files.txt\n> +++ b/Documentation/git-ls-files.txt\n> @@ -140,8 +140,7 @@ a space) at the start of each line:\n>\n> --recurse-submodules::\n> Recursively calls ls-files on each submodule in the repository.\n> - Currently there is only support for the --cached mode without a\n> - pathspec.\n> + Currently there is only support for the --cached.\n\ns/--cached/--cached mode/ ?\nThe \"the\" in front of --cached sounds a bit strange for a non native\nspeaker here.\n\n\n> + /*\n> + * Find common prefix for all pathspec's\n> + * This is used as a performance optimization which unfortunately cannot\n> + * be done when recursing into submodules\n> + */\n> + if (recurse_submodules)\n> + max_prefix = NULL;\n> + else\n> + max_prefix = common_prefix(&pathspec);\n\nNit of the day:\nWhile this is readable, you may want to explore how this reads shorter as\n\n max_prefix = recurse_submodules ? NULL : common_prefix(&pathspec);\n\n?\n\n> + git ls-files --recurse-submodules \"sub\" >actual &&\n> + test_cmp expect actual &&\n> + git ls-files --recurse-submodules \"sub/\" >actual &&\n> + test_cmp expect actual &&\n> + git ls-files --recurse-submodules \"sub/file\" >actual &&\n> + test_cmp expect actual &&\n> + git ls-files --recurse-submodules \"su*/file\" >actual &&\n> + test_cmp expect actual &&\n> + git ls-files --recurse-submodules \"su?/file\" >actual &&\n> + test_cmp expect actual\n> +'\n\n> +\n> + git ls-files --recurse-submodules \"s??/file\" >actual &&\n> + test_cmp expect actual &&\n> + git ls-files --recurse-submodules \"s???file\" >actual &&\n> + test_cmp expect actual &&\n> + git ls-files --recurse-submodules \"s*file\" >actual &&\n> + test_cmp expect actual\n> '\n\nThanks for the tests!\n"},{"id":"303598","messageId":"20161007181851.15063-1-bmwill@google.com","threadId":"44147","inReplyTo":"1475185723-36871-1-git-send-email-bmwill@google.com","subject":"[PATCH v7 0/4] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-10-07T18:18:47Z","receivedAt":"2016-10-07T18:19:19Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Few minor fixes pointed out Stefan\n\nBrandon Williams (4):\n git: make super-prefix option\n ls-files: optionally recurse into submodules\n ls-files: pass through safe options for --recurse-submodules\n ls-files: add pathspec matching for submodules\n\n Documentation/git-ls-files.txt | 7 +-\n Documentation/git.txt | 6 +\n builtin/ls-files.c | 181 +++++++++++++++++++++-------\n cache.h | 2 +\n dir.c | 46 +++++++-\n dir.h | 4 +\n environment.c | 13 ++\n git.c | 27 ++++-\n t/t3007-ls-files-recurse-submodules.sh | 210 +++++++++++++++++++++++++++++++++\n 9 files changed, 452 insertions(+), 44 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\n-- \n2.10.0\n\n"},{"id":"303599","messageId":"20161007181851.15063-2-bmwill@google.com","threadId":"44147","inReplyTo":"20161007181851.15063-1-bmwill@google.com","subject":"[PATCH v7 1/4] git: make super-prefix option","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-10-07T18:18:48Z","receivedAt":"2016-10-07T18:19:25Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Add a super-prefix environment variable 'GIT_INTERNAL_SUPER_PREFIX'\nwhich can be used to specify a path from above a repository down to its\nroot. When such a super-prefix is specified, the paths reported by Git\nare prefixed with it to make them relative to that directory \"above\".\nThe paths given by the user on the command line\n(e.g. \"git subcmd --output-file=path/to/a/file\" and pathspecs) are taken\nrelative to the directory \"above\" to match.\n\nThe immediate use of this option is by commands which have a\n--recurse-submodule option in order to give context to submodules about\nhow they were invoked. This option is currently only allowed for\nbuiltins which support a super-prefix.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git.txt | 6 ++++++\n cache.h | 2 ++\n environment.c | 13 +++++++++++++\n git.c | 25 +++++++++++++++++++++++++\n 4 files changed, 46 insertions(+)\n\ndiff --git a/Documentation/git.txt b/Documentation/git.txt\nindex 7913fc2..2188ae6 100644\n--- a/Documentation/git.txt\n+++ b/Documentation/git.txt\n@@ -13,6 +13,7 @@ SYNOPSIS\n [--exec-path[=]] [--html-path] [--man-path] [--info-path]\n [-p|--paginate|--no-pager] [--no-replace-objects] [--bare]\n [--git-dir=] [--work-tree=] [--namespace=]\n+ [--super-prefix=]\n []\n \n DESCRIPTION\n@@ -601,6 +602,11 @@ foo.bar= ...`) sets `foo.bar` to the empty string.\n \tdetails. Equivalent to setting the `GIT_NAMESPACE` environment\n \tvariable.\n \n+--super-prefix=::\n+\tCurrently for internal use only. Set a prefix which gives a path from\n+\tabove a repository down to its root. One use is to give submodules\n+\tcontext about the superproject that invoked it.\n+\n --bare::\n \tTreat the repository as a bare repository. If GIT_DIR\n \tenvironment is not set, it is set to the current working\ndiff --git a/cache.h b/cache.h\nindex 3556326..8cf495d 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -408,6 +408,7 @@ static inline enum object_type object_type(unsigned int mode)\n #define GIT_NAMESPACE_ENVIRONMENT \"GIT_NAMESPACE\"\n #define GIT_WORK_TREE_ENVIRONMENT \"GIT_WORK_TREE\"\n #define GIT_PREFIX_ENVIRONMENT \"GIT_PREFIX\"\n+#define GIT_SUPER_PREFIX_ENVIRONMENT \"GIT_INTERNAL_SUPER_PREFIX\"\n #define DEFAULT_GIT_DIR_ENVIRONMENT \".git\"\n #define DB_ENVIRONMENT \"GIT_OBJECT_DIRECTORY\"\n #define INDEX_ENVIRONMENT \"GIT_INDEX_FILE\"\n@@ -468,6 +469,7 @@ extern int get_common_dir_noenv(struct strbuf *sb, const char *gitdir);\n extern int get_common_dir(struct strbuf *sb, const char *gitdir);\n extern const char *get_git_namespace(void);\n extern const char *strip_namespace(const char *namespaced_ref);\n+extern const char *get_super_prefix(void);\n extern const char *get_git_work_tree(void);\n \n /*\ndiff --git a/environment.c b/environment.c\nindex ca72464..d12d7db 100644\n--- a/environment.c\n+++ b/environment.c\n@@ -100,6 +100,8 @@ static char *work_tree;\n static const char *namespace;\n static size_t namespace_len;\n \n+static const char *super_prefix;\n+\n static const char *git_dir, *git_common_dir;\n static char *git_object_dir, *git_index_file, *git_graft_file;\n int git_db_env, git_index_env, git_graft_env, git_common_dir_env;\n@@ -120,6 +122,7 @@ const char * const local_repo_env[] = {\n \tNO_REPLACE_OBJECTS_ENVIRONMENT,\n \tGIT_REPLACE_REF_BASE_ENVIRONMENT,\n \tGIT_PREFIX_ENVIRONMENT,\n+\tGIT_SUPER_PREFIX_ENVIRONMENT,\n \tGIT_SHALLOW_FILE_ENVIRONMENT,\n \tGIT_COMMON_DIR_ENVIRONMENT,\n \tNULL\n@@ -222,6 +225,16 @@ const char *strip_namespace(const char *namespaced_ref)\n \treturn namespaced_ref + namespace_len;\n }\n \n+const char *get_super_prefix(void)\n+{\n+\tstatic int initialized;\n+\tif (!initialized) {\n+\t\tsuper_prefix = getenv(GIT_SUPER_PREFIX_ENVIRONMENT);\n+\t\tinitialized = 1;\n+\t}\n+\treturn super_prefix;\n+}\n+\n static int git_work_tree_initialized;\n \n /*\ndiff --git a/git.c b/git.c\nindex 1c61151..469a83f 100644\n--- a/git.c\n+++ b/git.c\n@@ -164,6 +164,20 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)\n \t\t\tsetenv(GIT_WORK_TREE_ENVIRONMENT, cmd, 1);\n \t\t\tif (envchanged)\n \t\t\t\t*envchanged = 1;\n+\t\t} else if (!strcmp(cmd, \"--super-prefix\")) {\n+\t\t\tif (*argc < 2) {\n+\t\t\t\tfprintf(stderr, \"No prefix given for --super-prefix.\\n\" );\n+\t\t\t\tusage(git_usage_string);\n+\t\t\t}\n+\t\t\tsetenv(GIT_SUPER_PREFIX_ENVIRONMENT, (*argv)[1], 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n+\t\t\t(*argv)++;\n+\t\t\t(*argc)--;\n+\t\t} else if (skip_prefix(cmd, \"--super-prefix=\", &cmd)) {\n+\t\t\tsetenv(GIT_SUPER_PREFIX_ENVIRONMENT, cmd, 1);\n+\t\t\tif (envchanged)\n+\t\t\t\t*envchanged = 1;\n \t\t} else if (!strcmp(cmd, \"--bare\")) {\n \t\t\tchar *cwd = xgetcwd();\n \t\t\tis_bare_repository_cfg = 1;\n@@ -310,6 +324,7 @@ static int handle_alias(int *argcp, const char ***argv)\n * RUN_SETUP for reading from the configuration file.\n */\n #define NEED_WORK_TREE\t\t(1<<3)\n+#define SUPPORT_SUPER_PREFIX\t(1<<4)\n \n struct cmd_struct {\n \tconst char *cmd;\n@@ -344,6 +359,13 @@ static int run_builtin(struct cmd_struct *p, int argc, const char **argv)\n \t}\n \tcommit_pager_choice();\n \n+\tif (!help && get_super_prefix()) {\n+\t\tif (!(p->option & SUPPORT_SUPER_PREFIX))\n+\t\t\tdie(\"%s doesn't support --super-prefix\", p->cmd);\n+\t\tif (prefix)\n+\t\t\tdie(\"can't use --super-prefix from a subdirectory\");\n+\t}\n+\n \tif (!help && p->option & NEED_WORK_TREE)\n \t\tsetup_work_tree();\n \n@@ -558,6 +580,9 @@ static void execv_dashed_external(const char **argv)\n \tconst char *tmp;\n \tint status;\n \n+\tif (get_super_prefix())\n+\t\tdie(\"%s doesn't support --super-prefix\", argv[0]);\n+\n \tif (use_pager == -1)\n \t\tuse_pager = check_pager_config(argv[0]);\n \tcommit_pager_choice();\n-- \n2.10.0\n\n"},{"id":"303600","messageId":"20161007181851.15063-3-bmwill@google.com","threadId":"44147","inReplyTo":"20161007181851.15063-1-bmwill@google.com","subject":"[PATCH v7 2/4] ls-files: optionally recurse into submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-10-07T18:18:49Z","receivedAt":"2016-10-07T18:19:28Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Allow ls-files to recognize submodules in order to retrieve a list of\nfiles from a repository's submodules. This is done by forking off a\nprocess to recursively call ls-files on all submodules. Use top-level\n--super-prefix option to pass a path to the submodule which it can\nuse to prepend to output or pathspec matching logic.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git-ls-files.txt | 8 +-\n builtin/ls-files.c | 138 ++++++++++++++++++++++++---------\n git.c | 2 +-\n t/t3007-ls-files-recurse-submodules.sh | 100 ++++++++++++++++++++++++\n 4 files changed, 208 insertions(+), 40 deletions(-)\n create mode 100755 t/t3007-ls-files-recurse-submodules.sh\n\ndiff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\nindex 0d933ac..ea01d45 100644\n--- a/Documentation/git-ls-files.txt\n+++ b/Documentation/git-ls-files.txt\n@@ -18,7 +18,8 @@ SYNOPSIS\n \t\t[--exclude-per-directory=]\n \t\t[--exclude-standard]\n \t\t[--error-unmatch] [--with-tree=]\n-\t\t[--full-name] [--abbrev] [--] [...]\n+\t\t[--full-name] [--recurse-submodules]\n+\t\t[--abbrev] [--] [...]\n \n DESCRIPTION\n -----------\n@@ -137,6 +138,11 @@ a space) at the start of each line:\n \toption forces paths to be output relative to the project\n \ttop directory.\n \n+--recurse-submodules::\n+\tRecursively calls ls-files on each submodule in the repository.\n+\tCurrently there is only support for the --cached mode without a\n+\tpathspec.\n+\n --abbrev[=]::\n \tInstead of showing the full 40-byte hexadecimal object\n \tlines, show only a partial prefix.\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 00ea91a..63befed 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -14,6 +14,7 @@\n #include \"resolve-undo.h\"\n #include \"string-list.h\"\n #include \"pathspec.h\"\n+#include \"run-command.h\"\n \n static int abbrev;\n static int show_deleted;\n@@ -28,8 +29,10 @@ static int show_valid_bit;\n static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n+static int recurse_submodules;\n \n static const char *prefix;\n+static const char *super_prefix;\n static int max_prefix_len;\n static int prefix_len;\n static struct pathspec pathspec;\n@@ -68,11 +71,24 @@ static void write_eolinfo(const struct cache_entry *ce, const char *path)\n static void write_name(const char *name)\n {\n \t/*\n+\t * Prepend the super_prefix to name to construct the full_name to be\n+\t * written.\n+\t */\n+\tstruct strbuf full_name = STRBUF_INIT;\n+\tif (super_prefix) {\n+\t\tstrbuf_addstr(&full_name, super_prefix);\n+\t\tstrbuf_addstr(&full_name, name);\n+\t\tname = full_name.buf;\n+\t}\n+\n+\t/*\n \t * With \"--full-name\", prefix_len=0; this caller needs to pass\n \t * an empty string in that case (a NULL is good for \"\").\n \t */\n \twrite_name_quoted_relative(name, prefix_len ? prefix : NULL,\n \t\t\t\t stdout, line_terminator);\n+\n+\tstrbuf_release(&full_name);\n }\n \n static void show_dir_entry(const char *tag, struct dir_entry *ent)\n@@ -152,55 +168,84 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/**\n+ * Recursively call ls-files on a submodule\n+ */\n+static void show_gitlink(const struct cache_entry *ce)\n+{\n+\tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tint status;\n+\n+\targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n+\t\t\t super_prefix ? super_prefix : \"\",\n+\t\t\t ce->name);\n+\targv_array_push(&cp.args, \"ls-files\");\n+\targv_array_push(&cp.args, \"--recurse-submodules\");\n+\n+\tcp.git_cmd = 1;\n+\tcp.dir = ce->name;\n+\tstatus = run_command(&cp);\n+\tif (status)\n+\t\texit(status);\n+}\n+\n static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n {\n+\tstruct strbuf name = STRBUF_INIT;\n \tint len = max_prefix_len;\n+\tif (super_prefix)\n+\t\tstrbuf_addstr(&name, super_prefix);\n+\tstrbuf_addstr(&name, ce->name);\n \n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (!match_pathspec(&pathspec, ce->name, ce_namelen(ce),\n-\t\t\t len, ps_matched,\n-\t\t\t S_ISDIR(ce->ce_mode) || S_ISGITLINK(ce->ce_mode)))\n-\t\treturn;\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\t\tshow_gitlink(ce);\n+\t} else if (match_pathspec(&pathspec, name.buf, name.len,\n+\t\t\t\t len, ps_matched,\n+\t\t\t\t S_ISDIR(ce->ce_mode) ||\n+\t\t\t\t S_ISGITLINK(ce->ce_mode))) {\n+\t\tif (tag && *tag && show_valid_bit &&\n+\t\t (ce->ce_flags & CE_VALID)) {\n+\t\t\tstatic char alttag[4];\n+\t\t\tmemcpy(alttag, tag, 3);\n+\t\t\tif (isalpha(tag[0]))\n+\t\t\t\talttag[0] = tolower(tag[0]);\n+\t\t\telse if (tag[0] == '?')\n+\t\t\t\talttag[0] = '!';\n+\t\t\telse {\n+\t\t\t\talttag[0] = 'v';\n+\t\t\t\talttag[1] = tag[0];\n+\t\t\t\talttag[2] = ' ';\n+\t\t\t\talttag[3] = 0;\n+\t\t\t}\n+\t\t\ttag = alttag;\n+\t\t}\n \n-\tif (tag && *tag && show_valid_bit &&\n-\t (ce->ce_flags & CE_VALID)) {\n-\t\tstatic char alttag[4];\n-\t\tmemcpy(alttag, tag, 3);\n-\t\tif (isalpha(tag[0]))\n-\t\t\talttag[0] = tolower(tag[0]);\n-\t\telse if (tag[0] == '?')\n-\t\t\talttag[0] = '!';\n-\t\telse {\n-\t\t\talttag[0] = 'v';\n-\t\t\talttag[1] = tag[0];\n-\t\t\talttag[2] = ' ';\n-\t\t\talttag[3] = 0;\n+\t\tif (!show_stage) {\n+\t\t\tfputs(tag, stdout);\n+\t\t} else {\n+\t\t\tprintf(\"%s%06o %s %d\\t\",\n+\t\t\t tag,\n+\t\t\t ce->ce_mode,\n+\t\t\t find_unique_abbrev(ce->sha1,abbrev),\n+\t\t\t ce_stage(ce));\n+\t\t}\n+\t\twrite_eolinfo(ce, ce->name);\n+\t\twrite_name(ce->name);\n+\t\tif (debug_mode) {\n+\t\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n+\n+\t\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n+\t\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n+\t\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n+\t\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n+\t\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n \t\t}\n-\t\ttag = alttag;\n \t}\n \n-\tif (!show_stage) {\n-\t\tfputs(tag, stdout);\n-\t} else {\n-\t\tprintf(\"%s%06o %s %d\\t\",\n-\t\t tag,\n-\t\t ce->ce_mode,\n-\t\t find_unique_abbrev(ce->sha1,abbrev),\n-\t\t ce_stage(ce));\n-\t}\n-\twrite_eolinfo(ce, ce->name);\n-\twrite_name(ce->name);\n-\tif (debug_mode) {\n-\t\tconst struct stat_data *sd = &ce->ce_stat_data;\n-\n-\t\tprintf(\" ctime: %d:%d\\n\", sd->sd_ctime.sec, sd->sd_ctime.nsec);\n-\t\tprintf(\" mtime: %d:%d\\n\", sd->sd_mtime.sec, sd->sd_mtime.nsec);\n-\t\tprintf(\" dev: %d\\tino: %d\\n\", sd->sd_dev, sd->sd_ino);\n-\t\tprintf(\" uid: %d\\tgid: %d\\n\", sd->sd_uid, sd->sd_gid);\n-\t\tprintf(\" size: %d\\tflags: %x\\n\", sd->sd_size, ce->ce_flags);\n-\t}\n+\tstrbuf_release(&name);\n }\n \n static void show_ru_info(void)\n@@ -468,6 +513,8 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\t{ OPTION_SET_INT, 0, \"full-name\", &prefix_len, NULL,\n \t\t\tN_(\"make the output relative to the project top directory\"),\n \t\t\tPARSE_OPT_NOARG | PARSE_OPT_NONEG, NULL },\n+\t\tOPT_BOOL(0, \"recurse-submodules\", &recurse_submodules,\n+\t\t\tN_(\"recurse through submodules\")),\n \t\tOPT_BOOL(0, \"error-unmatch\", &error_unmatch,\n \t\t\tN_(\"if any is not in the index, treat this as an error\")),\n \t\tOPT_STRING(0, \"with-tree\", &with_tree, N_(\"tree-ish\"),\n@@ -484,6 +531,7 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tprefix = cmd_prefix;\n \tif (prefix)\n \t\tprefix_len = strlen(prefix);\n+\tsuper_prefix = get_super_prefix();\n \tgit_config(git_default_config, NULL);\n \n \tif (read_cache() < 0)\n@@ -519,6 +567,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules &&\n+\t (show_stage || show_deleted || show_others || show_unmerged ||\n+\t show_killed || show_modified || show_resolve_undo ||\n+\t show_valid_bit || show_tag || show_eol || with_tree ||\n+\t (line_terminator == '\\0')))\n+\t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n+\n+\tif (recurse_submodules && error_unmatch)\n+\t\tdie(\"ls-files --recurse-submodules does not support \"\n+\t\t \"--error-unmatch\");\n+\n+\tif (recurse_submodules && argc)\n+\t\tdie(\"ls-files --recurse-submodules does not support pathspec\");\n+\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\ndiff --git a/git.c b/git.c\nindex 469a83f..df73768 100644\n--- a/git.c\n+++ b/git.c\n@@ -443,7 +443,7 @@ static struct cmd_struct commands[] = {\n \t{ \"init-db\", cmd_init_db },\n \t{ \"interpret-trailers\", cmd_interpret_trailers, RUN_SETUP_GENTLY },\n \t{ \"log\", cmd_log, RUN_SETUP },\n-\t{ \"ls-files\", cmd_ls_files, RUN_SETUP },\n+\t{ \"ls-files\", cmd_ls_files, RUN_SETUP | SUPPORT_SUPER_PREFIX },\n \t{ \"ls-remote\", cmd_ls_remote, RUN_SETUP_GENTLY },\n \t{ \"ls-tree\", cmd_ls_tree, RUN_SETUP },\n \t{ \"mailinfo\", cmd_mailinfo },\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nnew file mode 100755\nindex 0000000..b5a53c3\n--- /dev/null\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -0,0 +1,100 @@\n+#!/bin/sh\n+\n+test_description='Test ls-files recurse-submodules feature\n+\n+This test verifies the recurse-submodules feature correctly lists files from\n+submodules.\n+'\n+\n+. ./test-lib.sh\n+\n+test_expect_success 'setup directory structure and submodules' '\n+\techo a >a &&\n+\tmkdir b &&\n+\techo b >b/b &&\n+\tgit add a b &&\n+\tgit commit -m \"add a and b\" &&\n+\tgit init submodule &&\n+\techo c >submodule/c &&\n+\tgit -C submodule add c &&\n+\tgit -C submodule commit -m \"add c\" &&\n+\tgit submodule add ./submodule &&\n+\tgit commit -m \"added submodule\"\n+'\n+\n+test_expect_success 'ls-files correctly outputs files in submodule' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files does not output files not added to a repo' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\techo a >not_added &&\n+\techo b >b/not_added &&\n+\techo c >submodule/not_added &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success 'ls-files recurses more than 1 level' '\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/subsub/d\n+\tEOF\n+\n+\tgit init submodule/subsub &&\n+\techo d >submodule/subsub/d &&\n+\tgit -C submodule/subsub add d &&\n+\tgit -C submodule/subsub commit -m \"add d\" &&\n+\tgit -C submodule submodule add ./subsub &&\n+\tgit -C submodule commit -m \"added subsub\" &&\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support using path arguments' '\n+\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n+\ttest_i18ngrep \"does not support pathspec\" actual\n+'\n+\n+test_expect_success '--recurse-submodules does not support --error-unmatch' '\n+\ttest_must_fail git ls-files --recurse-submodules --error-unmatch 2>actual &&\n+\ttest_i18ngrep \"does not support --error-unmatch\" actual\n+'\n+\n+test_incompatible_with_recurse_submodules () {\n+\ttest_expect_success \"--recurse-submodules and $1 are incompatible\" \"\n+\t\ttest_must_fail git ls-files --recurse-submodules $1 2>actual &&\n+\t\ttest_i18ngrep 'unsupported mode' actual\n+\t\"\n+}\n+\n+test_incompatible_with_recurse_submodules -z\n+test_incompatible_with_recurse_submodules -v\n+test_incompatible_with_recurse_submodules -t\n+test_incompatible_with_recurse_submodules --deleted\n+test_incompatible_with_recurse_submodules --modified\n+test_incompatible_with_recurse_submodules --others\n+test_incompatible_with_recurse_submodules --stage\n+test_incompatible_with_recurse_submodules --killed\n+test_incompatible_with_recurse_submodules --unmerged\n+test_incompatible_with_recurse_submodules --eol\n+\n+test_done\n-- \n2.10.0\n\n"},{"id":"303601","messageId":"20161007181851.15063-4-bmwill@google.com","threadId":"44147","inReplyTo":"20161007181851.15063-1-bmwill@google.com","subject":"[PATCH v7 3/4] ls-files: pass through safe options for --recurse-submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-10-07T18:18:50Z","receivedAt":"2016-10-07T18:19:32Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pass through some known-safe options when recursing into submodules.\n(--cached, -v, -t, -z, --debug, --eol)\n\nSigned-off-by: Brandon Williams \n---\n builtin/ls-files.c | 30 +++++++++++++++++++++++++++---\n t/t3007-ls-files-recurse-submodules.sh | 16 ++++++++++++----\n 2 files changed, 39 insertions(+), 7 deletions(-)\n\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex 63befed..b6144a5 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -30,6 +30,7 @@ static int line_terminator = '\\n';\n static int debug_mode;\n static int show_eol;\n static int recurse_submodules;\n+static struct argv_array submodules_options = ARGV_ARRAY_INIT;\n \n static const char *prefix;\n static const char *super_prefix;\n@@ -168,6 +169,25 @@ static void show_killed_files(struct dir_struct *dir)\n \t}\n }\n \n+/*\n+ * Compile an argv_array with all of the options supported by --recurse_submodules\n+ */\n+static void compile_submodule_options(const struct dir_struct *dir, int show_tag)\n+{\n+\tif (line_terminator == '\\0')\n+\t\targv_array_push(&submodules_options, \"-z\");\n+\tif (show_tag)\n+\t\targv_array_push(&submodules_options, \"-t\");\n+\tif (show_valid_bit)\n+\t\targv_array_push(&submodules_options, \"-v\");\n+\tif (show_cached)\n+\t\targv_array_push(&submodules_options, \"--cached\");\n+\tif (show_eol)\n+\t\targv_array_push(&submodules_options, \"--eol\");\n+\tif (debug_mode)\n+\t\targv_array_push(&submodules_options, \"--debug\");\n+}\n+\n /**\n * Recursively call ls-files on a submodule\n */\n@@ -182,6 +202,9 @@ static void show_gitlink(const struct cache_entry *ce)\n \targv_array_push(&cp.args, \"ls-files\");\n \targv_array_push(&cp.args, \"--recurse-submodules\");\n \n+\t/* add supported options */\n+\targv_array_pushv(&cp.args, submodules_options.argv);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -567,11 +590,12 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \tif (require_work_tree && !is_inside_work_tree())\n \t\tsetup_work_tree();\n \n+\tif (recurse_submodules)\n+\t\tcompile_submodule_options(&dir, show_tag);\n+\n \tif (recurse_submodules &&\n \t (show_stage || show_deleted || show_others || show_unmerged ||\n-\t show_killed || show_modified || show_resolve_undo ||\n-\t show_valid_bit || show_tag || show_eol || with_tree ||\n-\t (line_terminator == '\\0')))\n+\t show_killed || show_modified || show_resolve_undo || with_tree))\n \t\tdie(\"ls-files --recurse-submodules unsupported mode\");\n \n \tif (recurse_submodules && error_unmatch)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex b5a53c3..33a2ea7 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -34,6 +34,18 @@ test_expect_success 'ls-files correctly outputs files in submodule' '\n \ttest_cmp expect actual\n '\n \n+test_expect_success 'ls-files correctly outputs files in submodule with -z' '\n+\tlf_to_nul >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\tsubmodule/c\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules -z >actual &&\n+\ttest_cmp expect actual\n+'\n+\n test_expect_success 'ls-files does not output files not added to a repo' '\n \tcat >expect <<-\\EOF &&\n \t.gitmodules\n@@ -86,15 +98,11 @@ test_incompatible_with_recurse_submodules () {\n \t\"\n }\n \n-test_incompatible_with_recurse_submodules -z\n-test_incompatible_with_recurse_submodules -v\n-test_incompatible_with_recurse_submodules -t\n test_incompatible_with_recurse_submodules --deleted\n test_incompatible_with_recurse_submodules --modified\n test_incompatible_with_recurse_submodules --others\n test_incompatible_with_recurse_submodules --stage\n test_incompatible_with_recurse_submodules --killed\n test_incompatible_with_recurse_submodules --unmerged\n-test_incompatible_with_recurse_submodules --eol\n \n test_done\n-- \n2.10.0\n\n"},{"id":"303602","messageId":"20161007181851.15063-5-bmwill@google.com","threadId":"44147","inReplyTo":"20161007181851.15063-1-bmwill@google.com","subject":"[PATCH v7 4/4] ls-files: add pathspec matching for submodules","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-10-07T18:18:51Z","receivedAt":"2016-10-07T18:19:37Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"Pathspecs can be a bit tricky when trying to apply them to submodules.\nThe main challenge is that the pathspecs will be with respect to the\nsuperproject and not with respect to paths in the submodule. The\napproach this patch takes is to pass in the identical pathspec from the\nsuperproject to the submodule in addition to the submodule-prefix, which\nis the path from the root of the superproject to the submodule, and then\nwe can compare an entry in the submodule prepended with the\nsubmodule-prefix to the pathspec in order to determine if there is a\nmatch.\n\nThis patch also permits the pathspec logic to perform a prefix match against\nsubmodules since a pathspec could refer to a file inside of a submodule.\nDue to limitations in the wildmatch logic, a prefix match is only done\nliterally. If any wildcard character is encountered we'll simply punt\nand produce a false positive match. More accurate matching will be done\nonce inside the submodule. This is due to the superproject not knowing\nwhat files could exist in the submodule.\n\nSigned-off-by: Brandon Williams \n---\n Documentation/git-ls-files.txt | 3 +-\n builtin/ls-files.c | 27 +++++++--\n dir.c | 46 +++++++++++++-\n dir.h | 4 ++\n t/t3007-ls-files-recurse-submodules.sh | 108 ++++++++++++++++++++++++++++++++-\n 5 files changed, 175 insertions(+), 13 deletions(-)\n\ndiff --git a/Documentation/git-ls-files.txt b/Documentation/git-ls-files.txt\nindex ea01d45..446209e 100644\n--- a/Documentation/git-ls-files.txt\n+++ b/Documentation/git-ls-files.txt\n@@ -140,8 +140,7 @@ a space) at the start of each line:\n \n --recurse-submodules::\n \tRecursively calls ls-files on each submodule in the repository.\n-\tCurrently there is only support for the --cached mode without a\n-\tpathspec.\n+\tCurrently there is only support for the --cached mode.\n \n --abbrev[=]::\n \tInstead of showing the full 40-byte hexadecimal object\ndiff --git a/builtin/ls-files.c b/builtin/ls-files.c\nindex b6144a5..0f25914 100644\n--- a/builtin/ls-files.c\n+++ b/builtin/ls-files.c\n@@ -195,6 +195,7 @@ static void show_gitlink(const struct cache_entry *ce)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n \tint status;\n+\tint i;\n \n \targv_array_pushf(&cp.args, \"--super-prefix=%s%s/\",\n \t\t\t super_prefix ? super_prefix : \"\",\n@@ -205,6 +206,15 @@ static void show_gitlink(const struct cache_entry *ce)\n \t/* add supported options */\n \targv_array_pushv(&cp.args, submodules_options.argv);\n \n+\t/*\n+\t * Pass in the original pathspec args. The submodule will be\n+\t * responsible for prepending the 'submodule_prefix' prior to comparing\n+\t * against the pathspec for matches.\n+\t */\n+\targv_array_push(&cp.args, \"--\");\n+\tfor (i = 0; i < pathspec.nr; i++)\n+\t\targv_array_push(&cp.args, pathspec.items[i].original);\n+\n \tcp.git_cmd = 1;\n \tcp.dir = ce->name;\n \tstatus = run_command(&cp);\n@@ -223,7 +233,8 @@ static void show_ce_entry(const char *tag, const struct cache_entry *ce)\n \tif (len >= ce_namelen(ce))\n \t\tdie(\"git ls-files: internal error - cache entry not superset of prefix\");\n \n-\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode)) {\n+\tif (recurse_submodules && S_ISGITLINK(ce->ce_mode) &&\n+\t submodule_path_match(&pathspec, name.buf, ps_matched)) {\n \t\tshow_gitlink(ce);\n \t} else if (match_pathspec(&pathspec, name.buf, name.len,\n \t\t\t\t len, ps_matched,\n@@ -602,16 +613,20 @@ int cmd_ls_files(int argc, const char **argv, const char *cmd_prefix)\n \t\tdie(\"ls-files --recurse-submodules does not support \"\n \t\t \"--error-unmatch\");\n \n-\tif (recurse_submodules && argc)\n-\t\tdie(\"ls-files --recurse-submodules does not support pathspec\");\n-\n \tparse_pathspec(&pathspec, 0,\n \t\t PATHSPEC_PREFER_CWD |\n \t\t PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\n \t\t prefix, argv);\n \n-\t/* Find common prefix for all pathspec's */\n-\tmax_prefix = common_prefix(&pathspec);\n+\t/*\n+\t * Find common prefix for all pathspec's\n+\t * This is used as a performance optimization which unfortunately cannot\n+\t * be done when recursing into submodules\n+\t */\n+\tif (recurse_submodules)\n+\t\tmax_prefix = NULL;\n+\telse\n+\t\tmax_prefix = common_prefix(&pathspec);\n \tmax_prefix_len = max_prefix ? strlen(max_prefix) : 0;\n \n \t/* Treat unmatching pathspec elements as errors */\ndiff --git a/dir.c b/dir.c\nindex 0ea235f..28e9736 100644\n--- a/dir.c\n+++ b/dir.c\n@@ -207,8 +207,9 @@ int within_depth(const char *name, int namelen,\n \treturn 1;\n }\n \n-#define DO_MATCH_EXCLUDE 1\n-#define DO_MATCH_DIRECTORY 2\n+#define DO_MATCH_EXCLUDE (1<<0)\n+#define DO_MATCH_DIRECTORY (1<<1)\n+#define DO_MATCH_SUBMODULE (1<<2)\n \n /*\n * Does 'match' match the given name?\n@@ -283,6 +284,32 @@ static int match_pathspec_item(const struct pathspec_item *item, int prefix,\n \t\t\t item->nowildcard_len - prefix))\n \t\treturn MATCHED_FNMATCH;\n \n+\t/* Perform checks to see if \"name\" is a super set of the pathspec */\n+\tif (flags & DO_MATCH_SUBMODULE) {\n+\t\t/* name is a literal prefix of the pathspec */\n+\t\tif ((namelen < matchlen) &&\n+\t\t (match[namelen] == '/') &&\n+\t\t !ps_strncmp(item, match, name, namelen))\n+\t\t\treturn MATCHED_RECURSIVELY;\n+\n+\t\t/* name\" doesn't match up to the first wild character */\n+\t\tif (item->nowildcard_len < item->len &&\n+\t\t ps_strncmp(item, match, name,\n+\t\t\t item->nowildcard_len - prefix))\n+\t\t\treturn 0;\n+\n+\t\t/*\n+\t\t * Here is where we would perform a wildmatch to check if\n+\t\t * \"name\" can be matched as a directory (or a prefix) against\n+\t\t * the pathspec. Since wildmatch doesn't have this capability\n+\t\t * at the present we have to punt and say that it is a match,\n+\t\t * potentially returning a false positive\n+\t\t * The submodules themselves will be able to perform more\n+\t\t * accurate matching to determine if the pathspec matches.\n+\t\t */\n+\t\treturn MATCHED_RECURSIVELY;\n+\t}\n+\n \treturn 0;\n }\n \n@@ -386,6 +413,21 @@ int match_pathspec(const struct pathspec *ps,\n \treturn negative ? 0 : positive;\n }\n \n+/**\n+ * Check if a submodule is a superset of the pathspec\n+ */\n+int submodule_path_match(const struct pathspec *ps,\n+\t\t\t const char *submodule_name,\n+\t\t\t char *seen)\n+{\n+\tint matched = do_match_pathspec(ps, submodule_name,\n+\t\t\t\t\tstrlen(submodule_name),\n+\t\t\t\t\t0, seen,\n+\t\t\t\t\tDO_MATCH_DIRECTORY |\n+\t\t\t\t\tDO_MATCH_SUBMODULE);\n+\treturn matched;\n+}\n+\n int report_path_error(const char *ps_matched,\n \t\t const struct pathspec *pathspec,\n \t\t const char *prefix)\ndiff --git a/dir.h b/dir.h\nindex da1a858..97c83bb 100644\n--- a/dir.h\n+++ b/dir.h\n@@ -304,6 +304,10 @@ extern int git_fnmatch(const struct pathspec_item *item,\n \t\t const char *pattern, const char *string,\n \t\t int prefix);\n \n+extern int submodule_path_match(const struct pathspec *ps,\n+\t\t\t\tconst char *submodule_name,\n+\t\t\t\tchar *seen);\n+\n static inline int ce_path_match(const struct cache_entry *ce,\n \t\t\t\tconst struct pathspec *pathspec,\n \t\t\t\tchar *seen)\ndiff --git a/t/t3007-ls-files-recurse-submodules.sh b/t/t3007-ls-files-recurse-submodules.sh\nindex 33a2ea7..a542617 100755\n--- a/t/t3007-ls-files-recurse-submodules.sh\n+++ b/t/t3007-ls-files-recurse-submodules.sh\n@@ -81,9 +81,111 @@ test_expect_success 'ls-files recurses more than 1 level' '\n \ttest_cmp expect actual\n '\n \n-test_expect_success '--recurse-submodules does not support using path arguments' '\n-\ttest_must_fail git ls-files --recurse-submodules b 2>actual &&\n-\ttest_i18ngrep \"does not support pathspec\" actual\n+test_expect_success '--recurse-submodules and pathspecs setup' '\n+\techo e >submodule/subsub/e.txt &&\n+\tgit -C submodule/subsub add e.txt &&\n+\tgit -C submodule/subsub commit -m \"adding e.txt\" &&\n+\techo f >submodule/f.TXT &&\n+\techo g >submodule/g.txt &&\n+\tgit -C submodule add f.TXT g.txt &&\n+\tgit -C submodule commit -m \"add f and g\" &&\n+\techo h >h.txt &&\n+\tmkdir sib &&\n+\techo sib >sib/file &&\n+\tgit add h.txt sib/file &&\n+\tgit commit -m \"add h and sib/file\" &&\n+\tgit init sub &&\n+\techo sub >sub/file &&\n+\tgit -C sub add file &&\n+\tgit -C sub commit -m \"add file\" &&\n+\tgit submodule add ./sub &&\n+\tgit commit -m \"added sub\" &&\n+\n+\tcat >expect <<-\\EOF &&\n+\t.gitmodules\n+\ta\n+\tb/b\n+\th.txt\n+\tsib/file\n+\tsub/file\n+\tsubmodule/.gitmodules\n+\tsubmodule/c\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/d\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules >actual &&\n+\ttest_cmp expect actual &&\n+\tcat actual &&\n+\tgit ls-files --recurse-submodules \"*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tsubmodule/subsub/e.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\th.txt\n+\tsubmodule/f.TXT\n+\tsubmodule/g.txt\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \":(icase)*.txt\" \":(exclude)submodule/subsub/*\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"sub\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"sub/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su*/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"su?/file\" >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success '--recurse-submodules and pathspecs' '\n+\tcat >expect <<-\\EOF &&\n+\tsib/file\n+\tsub/file\n+\tEOF\n+\n+\tgit ls-files --recurse-submodules \"s??/file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s???file\" >actual &&\n+\ttest_cmp expect actual &&\n+\tgit ls-files --recurse-submodules \"s*file\" >actual &&\n+\ttest_cmp expect actual\n '\n \n test_expect_success '--recurse-submodules does not support --error-unmatch' '\n-- \n2.10.0\n\n"},{"id":"303604","messageId":"CAGZ79kacgsPg-dzx7JRyhA2bq9JMb2Cyg9PjWbnKi9ZMTPw7RQ@mail.gmail.com","threadId":"44147","inReplyTo":"20161007181851.15063-1-bmwill@google.com","subject":"Re: [PATCH v7 0/4] recursive support for ls-files","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-10-07T18:34:10Z","receivedAt":"2016-10-07T18:34:21Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Oct 7, 2016 at 11:18 AM, Brandon Williams wrote:\n> Few minor fixes pointed out Stefan\n\nNit: This is not very informative, so you could provide an \"interdiff\"\nto the last patch, (see https://github.com/trast/tbdiff for a\nsophisticated tool,\nI usually do a git diff origin/sb/)\nThat helps reviewers as they do not need to review it all again, but\nthey may remember what they annotated and see how you addressed it.\n\nThanks,\nStefan\n"},{"id":"303605","messageId":"CAGZ79kZo-hL5-rCdw2VA5ce+xpizbGUdkS8ddb6eMC8qQoRSVw@mail.gmail.com","threadId":"44147","inReplyTo":"CAGZ79kacgsPg-dzx7JRyhA2bq9JMb2Cyg9PjWbnKi9ZMTPw7RQ@mail.gmail.com","subject":"Re: [PATCH v7 0/4] recursive support for ls-files","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2016-10-07T18:35:55Z","receivedAt":"2016-10-07T18:36:46Z","isPatch":true,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"On Fri, Oct 7, 2016 at 11:34 AM, Stefan Beller wrote:\n> On Fri, Oct 7, 2016 at 11:18 AM, Brandon Williams wrote:\n>> Few minor fixes pointed out Stefan\n>\n\nThe series itself looks good though. :)\n\nThanks,\nStefan\n"},{"id":"303606","messageId":"20161007184503.GA16132@google.com","threadId":"44147","inReplyTo":"CAGZ79kZo-hL5-rCdw2VA5ce+xpizbGUdkS8ddb6eMC8qQoRSVw@mail.gmail.com","subject":"Re: [PATCH v7 0/4] recursive support for ls-files","fromName":"Brandon Williams","fromEmail":"bmwill@google.com","sentAt":"2016-10-07T18:45:03Z","receivedAt":"2016-10-07T18:45:11Z","isPatch":true,"sender":{"key":"bwilliams.eng@gmail.com","avatar":null},"body":"On 10/07, Stefan Beller wrote:\n> On Fri, Oct 7, 2016 at 11:34 AM, Stefan Beller wrote:\n> > On Fri, Oct 7, 2016 at 11:18 AM, Brandon Williams wrote:\n> >> Few minor fixes pointed out Stefan\n> >\n> \n> The series itself looks good though. :)\n> \n> Thanks,\n> Stefan\n\nThanks! And I'll keep that in mind for the future. Still lots to learn\nabout contributing to the project.\n\n-- \nBrandon Williams\n"}]}