{"thread":{"id":"40303","subject":"[PATCHv6 1/3] submodule: Reimplement `module_list` shell function in C","startedAt":"2015-09-08T18:57:42Z","lastAt":"2015-09-08T18:57:45Z","messageCount":4,"participants":["Stefan Beller"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"269626","messageId":"1441738665-29493-1-git-send-email-sbeller@google.com","threadId":"40303","inReplyTo":null,"subject":"[PATCHv6 0/3] submodule--helper: Have some refactoring only patches first","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-09-08T18:57:42Z","receivedAt":"2015-09-08T18:57:42Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Added suggestions from Eric and Junio. Interdiff to v5 below.\n\nStefan Beller (3):\n  submodule: Reimplement `module_list` shell function in C\n  submodule: Reimplement `module_name` shell function in C\n  submodule: Reimplement `module_clone` shell function in C\n\n .gitignore                  |   1 +\n Makefile                    |   1 +\n builtin.h                   |   1 +\n builtin/submodule--helper.c | 282 ++++++++++++++++++++++++++++++++++++++++++++\n git-submodule.sh            | 164 +++-----------------------\n git.c                       |   1 +\n 6 files changed, 301 insertions(+), 149 deletions(-)\n create mode 100644 builtin/submodule--helper.c\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 4e30d8e..f4c3eff 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -10,12 +10,16 @@\n #include \"string-list.h\"\n #include \"run-command.h\"\n \n-static const struct cache_entry **ce_entries;\n-static int ce_alloc, ce_used;\n+struct module_list {\n+\tconst struct cache_entry **entries;\n+\tint alloc, nr;\n+};\n+#define MODULE_LIST_INIT { NULL, 0, 0 }\n \n static int module_list_compute(int argc, const char **argv,\n-\t\t\t\tconst char *prefix,\n-\t\t\t\tstruct pathspec *pathspec)\n+\t\t\t       const char *prefix,\n+\t\t\t       struct pathspec *pathspec,\n+\t\t\t       struct module_list *list)\n {\n \tint i, result = 0;\n \tchar *max_prefix, *ps_matched = NULL;\n@@ -40,12 +44,11 @@ static int module_list_compute(int argc, const char **argv,\n \n \t\tif (!S_ISGITLINK(ce->ce_mode) ||\n \t\t    !match_pathspec(pathspec, ce->name, ce_namelen(ce),\n-\t\t\t\t    max_prefix_len, ps_matched,\n-\t\t\t\t    S_ISGITLINK(ce->ce_mode) | S_ISDIR(ce->ce_mode)))\n+\t\t\t\t    max_prefix_len, ps_matched, 1))\n \t\t\tcontinue;\n \n-\t\tALLOC_GROW(ce_entries, ce_used + 1, ce_alloc);\n-\t\tce_entries[ce_used++] = ce;\n+\t\tALLOC_GROW(list->entries, list->nr + 1, list->alloc);\n+\t\tlist->entries[list->nr++] = ce;\n \t\twhile (i + 1 < active_nr &&\n \t\t       !strcmp(ce->name, active_cache[i + 1]->name))\n \t\t\t/*\n@@ -68,6 +71,7 @@ static int module_list(int argc, const char **argv, const char *prefix)\n {\n \tint i;\n \tstruct pathspec pathspec;\n+\tstruct module_list list = MODULE_LIST_INIT;\n \n \tstruct option module_list_options[] = {\n \t\tOPT_STRING(0, \"prefix\", &prefix,\n@@ -84,13 +88,13 @@ static int module_list(int argc, const char **argv, const char *prefix)\n \targc = parse_options(argc, argv, prefix, module_list_options,\n \t\t\t     git_submodule_helper_usage, 0);\n \n-\tif (module_list_compute(argc, argv, prefix, &pathspec) < 0) {\n+\tif (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0) {\n \t\tprintf(\"#unmatched\\n\");\n \t\treturn 1;\n \t}\n \n-\tfor (i = 0; i < ce_used; i++) {\n-\t\tconst struct cache_entry *ce = ce_entries[i];\n+\tfor (i = 0; i < list.nr; i++) {\n+\t\tconst struct cache_entry *ce = list.entries[i];\n \n \t\tif (ce_stage(ce))\n \t\t\tprintf(\"%06o %s U\\t\", ce->ce_mode, sha1_to_hex(null_sha1));\n@@ -142,10 +146,7 @@ static int clone_submodule(const char *path, const char *gitdir, const char *url\n \n \tcp.git_cmd = 1;\n \tcp.env = local_repo_env;\n-\n \tcp.no_stdin = 1;\n-\tcp.no_stdout = 1;\n-\tcp.no_stderr = 1;\n \n \treturn run_command(&cp);\n }\n@@ -206,8 +207,7 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \t\tif (safe_create_leading_directories_const(path) < 0)\n \t\t\tdie(_(\"could not create directory '%s'\"), path);\n \t\tstrbuf_addf(&sb, \"%s/index\", sm_gitdir);\n-\t\tif (unlink(sb.buf) < 0)\n-\t\t\tdie_errno(_(\"failed to delete '%s'\"), sm_gitdir);\n+\t\tunlink_or_warn(sb.buf);\n \t\tstrbuf_reset(&sb);\n \t}\n \n@@ -218,7 +218,7 @@ static int module_clone(int argc, const char **argv, const char *prefix)\n \tif (path && *path)\n \t\tstrbuf_addf(&sb, \"%s/.git\", path);\n \telse\n-\t\tstrbuf_addf(&sb, \".git\");\n+\t\tstrbuf_addstr(&sb, \".git\");\n \n \tif (safe_create_leading_directories_const(sb.buf) < 0)\n \t\tdie(_(\"could not create leading directories of '%s'\"), sb.buf);\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 7cfdc2c..8b0eb9a 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -227,7 +227,7 @@ cmd_add()\n \t\t\tshift\n \t\t\t;;\n \t\t--depth=*)\n-\t\t\tdepth=\"$1\"\n+\t\t\tdepth=$1\n \t\t\t;;\n \t\t--)\n \t\t\tshift\n\n-- \n2.5.0.256.g89f8063.dirty\n"},{"id":"269623","messageId":"1441738665-29493-2-git-send-email-sbeller@google.com","threadId":"40303","inReplyTo":"1441738665-29493-1-git-send-email-sbeller@google.com","subject":"[PATCHv6 1/3] submodule: Reimplement `module_list` shell function in C","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-09-08T18:57:43Z","receivedAt":"2015-09-08T18:57:43Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"Most of the submodule operations work on a set of submodules.\nCalculating and using this set is usually done via:\n\n       module_list \"$@\" | {\n           while read mode sha1 stage sm_path\n           do\n                # the actual operation\n           done\n       }\n\nCurrently the function `module_list` is implemented in the\ngit-submodule.sh as a shell script wrapping a perl script.\nThe rewrite is in C, such that it is faster and can later be\neasily adapted when other functions are rewritten in C.\n\ngit-submodule.sh, similar to the builtin commands, will navigate\nto the top-most directory of the repository and keep the\nsubdirectory as a variable. As the helper is called from\nwithin the git-submodule.sh script, we are already navigated\nto the root level, but the path arguments are still relative\nto the subdirectory we were in when calling git-submodule.sh.\nThat's why there is a `--prefix` option pointing to an alternative\npath which to anchor relative path arguments.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n .gitignore                  |   1 +\n Makefile                    |   1 +\n builtin.h                   |   1 +\n builtin/submodule--helper.c | 128 ++++++++++++++++++++++++++++++++++++++++++++\n git-submodule.sh            |  54 +++----------------\n git.c                       |   1 +\n 6 files changed, 138 insertions(+), 48 deletions(-)\n create mode 100644 builtin/submodule--helper.c\n\ndiff --git a/.gitignore b/.gitignore\nindex 4fd81ba..1c2f832 100644\n--- a/.gitignore\n+++ b/.gitignore\n@@ -155,6 +155,7 @@\n /git-status\n /git-stripspace\n /git-submodule\n+/git-submodule--helper\n /git-svn\n /git-symbolic-ref\n /git-tag\ndiff --git a/Makefile b/Makefile\nindex 24b636d..d434e73 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -901,6 +901,7 @@ BUILTIN_OBJS += builtin/shortlog.o\n BUILTIN_OBJS += builtin/show-branch.o\n BUILTIN_OBJS += builtin/show-ref.o\n BUILTIN_OBJS += builtin/stripspace.o\n+BUILTIN_OBJS += builtin/submodule--helper.o\n BUILTIN_OBJS += builtin/symbolic-ref.o\n BUILTIN_OBJS += builtin/tag.o\n BUILTIN_OBJS += builtin/unpack-file.o\ndiff --git a/builtin.h b/builtin.h\nindex 839483d..924e6c4 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -119,6 +119,7 @@ extern int cmd_show(int argc, const char **argv, const char *prefix);\n extern int cmd_show_branch(int argc, const char **argv, const char *prefix);\n extern int cmd_status(int argc, const char **argv, const char *prefix);\n extern int cmd_stripspace(int argc, const char **argv, const char *prefix);\n+extern int cmd_submodule__helper(int argc, const char **argv, const char *prefix);\n extern int cmd_symbolic_ref(int argc, const char **argv, const char *prefix);\n extern int cmd_tag(int argc, const char **argv, const char *prefix);\n extern int cmd_tar_tree(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nnew file mode 100644\nindex 0000000..10db4e6\n--- /dev/null\n+++ b/builtin/submodule--helper.c\n@@ -0,0 +1,128 @@\n+#include \"builtin.h\"\n+#include \"cache.h\"\n+#include \"parse-options.h\"\n+#include \"quote.h\"\n+#include \"pathspec.h\"\n+#include \"dir.h\"\n+#include \"utf8.h\"\n+\n+struct module_list {\n+\tconst struct cache_entry **entries;\n+\tint alloc, nr;\n+};\n+#define MODULE_LIST_INIT { NULL, 0, 0 }\n+\n+static int module_list_compute(int argc, const char **argv,\n+\t\t\t       const char *prefix,\n+\t\t\t       struct pathspec *pathspec,\n+\t\t\t       struct module_list *list)\n+{\n+\tint i, result = 0;\n+\tchar *max_prefix, *ps_matched = NULL;\n+\tint max_prefix_len;\n+\tparse_pathspec(pathspec, 0,\n+\t\t       PATHSPEC_PREFER_FULL |\n+\t\t       PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP,\n+\t\t       prefix, argv);\n+\n+\t/* Find common prefix for all pathspec's */\n+\tmax_prefix = common_prefix(pathspec);\n+\tmax_prefix_len = max_prefix ? strlen(max_prefix) : 0;\n+\n+\tif (pathspec->nr)\n+\t\tps_matched = xcalloc(pathspec->nr, 1);\n+\n+\tif (read_cache() < 0)\n+\t\tdie(_(\"index file corrupt\"));\n+\n+\tfor (i = 0; i < active_nr; i++) {\n+\t\tconst struct cache_entry *ce = active_cache[i];\n+\n+\t\tif (!S_ISGITLINK(ce->ce_mode) ||\n+\t\t    !match_pathspec(pathspec, ce->name, ce_namelen(ce),\n+\t\t\t\t    max_prefix_len, ps_matched, 1))\n+\t\t\tcontinue;\n+\n+\t\tALLOC_GROW(list->entries, list->nr + 1, list->alloc);\n+\t\tlist->entries[list->nr++] = ce;\n+\t\twhile (i + 1 < active_nr &&\n+\t\t       !strcmp(ce->name, active_cache[i + 1]->name))\n+\t\t\t/*\n+\t\t\t * Skip entries with the same name in different stages\n+\t\t\t * to make sure an entry is returned only once.\n+\t\t\t */\n+\t\t\ti++;\n+\t}\n+\tfree(max_prefix);\n+\n+\tif (ps_matched && report_path_error(ps_matched, pathspec, prefix))\n+\t\tresult = -1;\n+\n+\tfree(ps_matched);\n+\n+\treturn result;\n+}\n+\n+static int module_list(int argc, const char **argv, const char *prefix)\n+{\n+\tint i;\n+\tstruct pathspec pathspec;\n+\tstruct module_list list = MODULE_LIST_INIT;\n+\n+\tstruct option module_list_options[] = {\n+\t\tOPT_STRING(0, \"prefix\", &prefix,\n+\t\t\t   N_(\"path\"),\n+\t\t\t   N_(\"alternative anchor for relative paths\")),\n+\t\tOPT_END()\n+\t};\n+\n+\tconst char *const git_submodule_helper_usage[] = {\n+\t\tN_(\"git submodule--helper list [--prefix=<path>] [<path>...]\"),\n+\t\tNULL\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, module_list_options,\n+\t\t\t     git_submodule_helper_usage, 0);\n+\n+\tif (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0) {\n+\t\tprintf(\"#unmatched\\n\");\n+\t\treturn 1;\n+\t}\n+\n+\tfor (i = 0; i < list.nr; i++) {\n+\t\tconst struct cache_entry *ce = list.entries[i];\n+\n+\t\tif (ce_stage(ce))\n+\t\t\tprintf(\"%06o %s U\\t\", ce->ce_mode, sha1_to_hex(null_sha1));\n+\t\telse\n+\t\t\tprintf(\"%06o %s %d\\t\", ce->ce_mode, sha1_to_hex(ce->sha1), ce_stage(ce));\n+\n+\t\tutf8_fprintf(stdout, \"%s\\n\", ce->name);\n+\t}\n+\treturn 0;\n+}\n+\n+\n+struct cmd_struct {\n+\tconst char *cmd;\n+\tint (*fn)(int, const char **, const char *);\n+};\n+\n+static struct cmd_struct commands[] = {\n+\t{\"list\", module_list},\n+};\n+\n+int cmd_submodule__helper(int argc, const char **argv, const char *prefix)\n+{\n+\tint i;\n+\tif (argc < 2)\n+\t\tdie(_(\"fatal: submodule--helper subcommand must be \"\n+\t\t      \"called with a subcommand\"));\n+\n+\tfor (i = 0; i < ARRAY_SIZE(commands); i++)\n+\t\tif (!strcmp(argv[1], commands[i].cmd))\n+\t\t\treturn commands[i].fn(argc - 1, argv + 1, prefix);\n+\n+\tdie(_(\"fatal: '%s' is not a valid submodule--helper \"\n+\t      \"subcommand\"), argv[1]);\n+}\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 36797c3..95c04fc 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -145,48 +145,6 @@ relative_path ()\n \techo \"$result$target\"\n }\n \n-#\n-# Get submodule info for registered submodules\n-# $@ = path to limit submodule list\n-#\n-module_list()\n-{\n-\teval \"set $(git rev-parse --sq --prefix \"$wt_prefix\" -- \"$@\")\"\n-\t(\n-\t\tgit ls-files -z --error-unmatch --stage -- \"$@\" ||\n-\t\techo \"unmatched pathspec exists\"\n-\t) |\n-\t@@PERL@@ -e '\n-\tmy %unmerged = ();\n-\tmy ($null_sha1) = (\"0\" x 40);\n-\tmy @out = ();\n-\tmy $unmatched = 0;\n-\t$/ = \"\\0\";\n-\twhile (<STDIN>) {\n-\t\tif (/^unmatched pathspec/) {\n-\t\t\t$unmatched = 1;\n-\t\t\tnext;\n-\t\t}\n-\t\tchomp;\n-\t\tmy ($mode, $sha1, $stage, $path) =\n-\t\t\t/^([0-7]+) ([0-9a-f]{40}) ([0-3])\\t(.*)$/;\n-\t\tnext unless $mode eq \"160000\";\n-\t\tif ($stage ne \"0\") {\n-\t\t\tif (!$unmerged{$path}++) {\n-\t\t\t\tpush @out, \"$mode $null_sha1 U\\t$path\\n\";\n-\t\t\t}\n-\t\t\tnext;\n-\t\t}\n-\t\tpush @out, \"$_\\n\";\n-\t}\n-\tif ($unmatched) {\n-\t\tprint \"#unmatched\\n\";\n-\t} else {\n-\t\tprint for (@out);\n-\t}\n-\t'\n-}\n-\n die_if_unmatched ()\n {\n \tif test \"$1\" = \"#unmatched\"\n@@ -532,7 +490,7 @@ cmd_foreach()\n \t# command in the subshell (and a recursive call to this function)\n \texec 3<&0\n \n-\tmodule_list |\n+\tgit submodule--helper list --prefix \"$wt_prefix\"|\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n@@ -592,7 +550,7 @@ cmd_init()\n \t\tshift\n \tdone\n \n-\tmodule_list \"$@\" |\n+\tgit submodule--helper list --prefix \"$wt_prefix\" \"$@\" |\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n@@ -674,7 +632,7 @@ cmd_deinit()\n \t\tdie \"$(eval_gettext \"Use '.' if you really want to deinitialize all submodules\")\"\n \tfi\n \n-\tmodule_list \"$@\" |\n+\tgit submodule--helper list --prefix \"$wt_prefix\" \"$@\" |\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n@@ -790,7 +748,7 @@ cmd_update()\n \tfi\n \n \tcloned_modules=\n-\tmodule_list \"$@\" | {\n+\tgit submodule--helper list --prefix \"$wt_prefix\" \"$@\" | {\n \terr=\n \twhile read mode sha1 stage sm_path\n \tdo\n@@ -1222,7 +1180,7 @@ cmd_status()\n \t\tshift\n \tdone\n \n-\tmodule_list \"$@\" |\n+\tgit submodule--helper list --prefix \"$wt_prefix\" \"$@\" |\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n@@ -1299,7 +1257,7 @@ cmd_sync()\n \t\tesac\n \tdone\n \tcd_to_toplevel\n-\tmodule_list \"$@\" |\n+\tgit submodule--helper list --prefix \"$wt_prefix\" \"$@\" |\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\ndiff --git a/git.c b/git.c\nindex 55c327c..deecba0 100644\n--- a/git.c\n+++ b/git.c\n@@ -469,6 +469,7 @@ static struct cmd_struct commands[] = {\n \t{ \"stage\", cmd_add, RUN_SETUP | NEED_WORK_TREE },\n \t{ \"status\", cmd_status, RUN_SETUP | NEED_WORK_TREE },\n \t{ \"stripspace\", cmd_stripspace },\n+\t{ \"submodule--helper\", cmd_submodule__helper, RUN_SETUP },\n \t{ \"symbolic-ref\", cmd_symbolic_ref, RUN_SETUP },\n \t{ \"tag\", cmd_tag, RUN_SETUP },\n \t{ \"unpack-file\", cmd_unpack_file, RUN_SETUP },\n-- \n2.5.0.256.g89f8063.dirty\n"},{"id":"269624","messageId":"1441738665-29493-3-git-send-email-sbeller@google.com","threadId":"40303","inReplyTo":"1441738665-29493-1-git-send-email-sbeller@google.com","subject":"[PATCHv6 2/3] submodule: Reimplement `module_name` shell function in C","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-09-08T18:57:44Z","receivedAt":"2015-09-08T18:57:44Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This implements the helper `name` in C instead of shell,\nyielding a nice performance boost.\n\nBefore this patch, I measured a time (best out of three):\n\n  $ time ./t7400-submodule-basic.sh  >/dev/null\n    real\t0m11.066s\n    user\t0m3.348s\n    sys\t0m8.534s\n\nWith this patch applied I measured (also best out of three)\n\n  $ time ./t7400-submodule-basic.sh  >/dev/null\n    real\t0m10.063s\n    user\t0m3.044s\n    sys\t0m7.487s\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 22 ++++++++++++++++++++++\n git-submodule.sh            | 32 +++++++-------------------------\n 2 files changed, 29 insertions(+), 25 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex 10db4e6..bc79c41 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -5,6 +5,9 @@\n #include \"pathspec.h\"\n #include \"dir.h\"\n #include \"utf8.h\"\n+#include \"submodule.h\"\n+#include \"submodule-config.h\"\n+#include \"string-list.h\"\n \n struct module_list {\n \tconst struct cache_entry **entries;\n@@ -102,6 +105,24 @@ static int module_list(int argc, const char **argv, const char *prefix)\n \treturn 0;\n }\n \n+static int module_name(int argc, const char **argv, const char *prefix)\n+{\n+\tconst struct submodule *sub;\n+\n+\tif (argc != 2)\n+\t\tusage(_(\"git submodule--helper name <path>\"));\n+\n+\tgitmodules_config();\n+\tsub = submodule_from_path(null_sha1, argv[1]);\n+\n+\tif (!sub)\n+\t\tdie(_(\"no submodule mapping found in .gitmodules for path '%s'\"),\n+\t\t    argv[1]);\n+\n+\tprintf(\"%s\\n\", sub->name);\n+\n+\treturn 0;\n+}\n \n struct cmd_struct {\n \tconst char *cmd;\n@@ -110,6 +131,7 @@ struct cmd_struct {\n \n static struct cmd_struct commands[] = {\n \t{\"list\", module_list},\n+\t{\"name\", module_name},\n };\n \n int cmd_submodule__helper(int argc, const char **argv, const char *prefix)\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 95c04fc..2be8da2 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -178,24 +178,6 @@ get_submodule_config () {\n \tprintf '%s' \"${value:-$default}\"\n }\n \n-\n-#\n-# Map submodule path to submodule name\n-#\n-# $1 = path\n-#\n-module_name()\n-{\n-\t# Do we have \"submodule.<something>.path = $1\" defined in .gitmodules file?\n-\tsm_path=\"$1\"\n-\tre=$(printf '%s\\n' \"$1\" | sed -e 's/[].[^$\\\\*]/\\\\&/g')\n-\tname=$( git config -f .gitmodules --get-regexp '^submodule\\..*\\.path$' |\n-\t\tsed -n -e 's|^submodule\\.\\(.*\\)\\.path '\"$re\"'$|\\1|p' )\n-\ttest -z \"$name\" &&\n-\tdie \"$(eval_gettext \"No submodule mapping found in .gitmodules for path '\\$sm_path'\")\"\n-\tprintf '%s\\n' \"$name\"\n-}\n-\n #\n # Clone a submodule\n #\n@@ -498,7 +480,7 @@ cmd_foreach()\n \t\tthen\n \t\t\tdisplaypath=$(relative_path \"$sm_path\")\n \t\t\tsay \"$(eval_gettext \"Entering '\\$prefix\\$displaypath'\")\"\n-\t\t\tname=$(module_name \"$sm_path\")\n+\t\t\tname=$(git submodule--helper name \"$sm_path\")\n \t\t\t(\n \t\t\t\tprefix=\"$prefix$sm_path/\"\n \t\t\t\tclear_local_git_env\n@@ -554,7 +536,7 @@ cmd_init()\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n-\t\tname=$(module_name \"$sm_path\") || exit\n+\t\tname=$(git submodule--helper name \"$sm_path\") || exit\n \n \t\tdisplaypath=$(relative_path \"$sm_path\")\n \n@@ -636,7 +618,7 @@ cmd_deinit()\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n-\t\tname=$(module_name \"$sm_path\") || exit\n+\t\tname=$(git submodule--helper name \"$sm_path\") || exit\n \n \t\tdisplaypath=$(relative_path \"$sm_path\")\n \n@@ -758,7 +740,7 @@ cmd_update()\n \t\t\techo >&2 \"Skipping unmerged submodule $prefix$sm_path\"\n \t\t\tcontinue\n \t\tfi\n-\t\tname=$(module_name \"$sm_path\") || exit\n+\t\tname=$(git submodule--helper name \"$sm_path\") || exit\n \t\turl=$(git config submodule.\"$name\".url)\n \t\tbranch=$(get_submodule_config \"$name\" branch master)\n \t\tif ! test -z \"$update\"\n@@ -1022,7 +1004,7 @@ cmd_summary() {\n \t\t\t# Respect the ignore setting for --for-status.\n \t\t\tif test -n \"$for_status\"\n \t\t\tthen\n-\t\t\t\tname=$(module_name \"$sm_path\")\n+\t\t\t\tname=$(git submodule--helper name \"$sm_path\")\n \t\t\t\tignore_config=$(get_submodule_config \"$name\" ignore none)\n \t\t\t\ttest $status != A && test $ignore_config = all && continue\n \t\t\tfi\n@@ -1184,7 +1166,7 @@ cmd_status()\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n-\t\tname=$(module_name \"$sm_path\") || exit\n+\t\tname=$(git submodule--helper name \"$sm_path\") || exit\n \t\turl=$(git config submodule.\"$name\".url)\n \t\tdisplaypath=$(relative_path \"$prefix$sm_path\")\n \t\tif test \"$stage\" = U\n@@ -1261,7 +1243,7 @@ cmd_sync()\n \twhile read mode sha1 stage sm_path\n \tdo\n \t\tdie_if_unmatched \"$mode\"\n-\t\tname=$(module_name \"$sm_path\")\n+\t\tname=$(git submodule--helper name \"$sm_path\")\n \t\turl=$(git config -f .gitmodules --get submodule.\"$name\".url)\n \n \t\t# Possibly a url relative to parent\n-- \n2.5.0.256.g89f8063.dirty\n"},{"id":"269625","messageId":"1441738665-29493-4-git-send-email-sbeller@google.com","threadId":"40303","inReplyTo":"1441738665-29493-1-git-send-email-sbeller@google.com","subject":"[PATCHv6 3/3] submodule: Reimplement `module_clone` shell function in C","fromName":"Stefan Beller","fromEmail":"sbeller@google.com","sentAt":"2015-09-08T18:57:45Z","receivedAt":"2015-09-08T18:57:45Z","isPatch":false,"sender":{"key":"stefanbeller@gmail.com","avatar":"https://avatars.githubusercontent.com/u/455868?v=4"},"body":"This reimplements the helper function `module_clone` in shell\nin C as `clone`. This functionality is needed for converting\n`git submodule update` later on, which we want to add threading\nto.\n\nSigned-off-by: Stefan Beller <sbeller@google.com>\n---\n builtin/submodule--helper.c | 132 ++++++++++++++++++++++++++++++++++++++++++++\n git-submodule.sh            |  78 +-------------------------\n 2 files changed, 134 insertions(+), 76 deletions(-)\n\ndiff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c\nindex bc79c41..f4c3eff 100644\n--- a/builtin/submodule--helper.c\n+++ b/builtin/submodule--helper.c\n@@ -8,6 +8,7 @@\n #include \"submodule.h\"\n #include \"submodule-config.h\"\n #include \"string-list.h\"\n+#include \"run-command.h\"\n \n struct module_list {\n \tconst struct cache_entry **entries;\n@@ -123,6 +124,136 @@ static int module_name(int argc, const char **argv, const char *prefix)\n \n \treturn 0;\n }\n+static int clone_submodule(const char *path, const char *gitdir, const char *url,\n+\t\t\t   const char *depth, const char *reference, int quiet)\n+{\n+\tstruct child_process cp;\n+\tchild_process_init(&cp);\n+\n+\targv_array_push(&cp.args, \"clone\");\n+\targv_array_push(&cp.args, \"--no-checkout\");\n+\tif (quiet)\n+\t\targv_array_push(&cp.args, \"--quiet\");\n+\tif (depth && *depth)\n+\t\targv_array_pushl(&cp.args, \"--depth\", depth, NULL);\n+\tif (reference && *reference)\n+\t\targv_array_pushl(&cp.args, \"--reference\", reference, NULL);\n+\tif (gitdir && *gitdir)\n+\t\targv_array_pushl(&cp.args, \"--separate-git-dir\", gitdir, NULL);\n+\n+\targv_array_push(&cp.args, url);\n+\targv_array_push(&cp.args, path);\n+\n+\tcp.git_cmd = 1;\n+\tcp.env = local_repo_env;\n+\tcp.no_stdin = 1;\n+\n+\treturn run_command(&cp);\n+}\n+\n+static int module_clone(int argc, const char **argv, const char *prefix)\n+{\n+\tconst char *path = NULL, *name = NULL, *url = NULL;\n+\tconst char *reference = NULL, *depth = NULL;\n+\tint quiet = 0;\n+\tFILE *submodule_dot_git;\n+\tchar *sm_gitdir, *cwd, *p;\n+\tstruct strbuf rel_path = STRBUF_INIT;\n+\tstruct strbuf sb = STRBUF_INIT;\n+\n+\tstruct option module_clone_options[] = {\n+\t\tOPT_STRING(0, \"prefix\", &prefix,\n+\t\t\t   N_(\"path\"),\n+\t\t\t   N_(\"alternative anchor for relative paths\")),\n+\t\tOPT_STRING(0, \"path\", &path,\n+\t\t\t   N_(\"path\"),\n+\t\t\t   N_(\"where the new submodule will be cloned to\")),\n+\t\tOPT_STRING(0, \"name\", &name,\n+\t\t\t   N_(\"string\"),\n+\t\t\t   N_(\"name of the new submodule\")),\n+\t\tOPT_STRING(0, \"url\", &url,\n+\t\t\t   N_(\"string\"),\n+\t\t\t   N_(\"url where to clone the submodule from\")),\n+\t\tOPT_STRING(0, \"reference\", &reference,\n+\t\t\t   N_(\"string\"),\n+\t\t\t   N_(\"reference repository\")),\n+\t\tOPT_STRING(0, \"depth\", &depth,\n+\t\t\t   N_(\"string\"),\n+\t\t\t   N_(\"depth for shallow clones\")),\n+\t\tOPT__QUIET(&quiet, \"Suppress output for cloning a submodule\"),\n+\t\tOPT_END()\n+\t};\n+\n+\tconst char *const git_submodule_helper_usage[] = {\n+\t\tN_(\"git submodule--helper clone [--prefix=<path>] [--quiet] \"\n+\t\t   \"[--reference <repository>] [--name <name>] [--url <url>]\"\n+\t\t   \"[--depth <depth>] [--] [<path>...]\"),\n+\t\tNULL\n+\t};\n+\n+\targc = parse_options(argc, argv, prefix, module_clone_options,\n+\t\t\t     git_submodule_helper_usage, 0);\n+\n+\tstrbuf_addf(&sb, \"%s/modules/%s\", get_git_dir(), name);\n+\tsm_gitdir = strbuf_detach(&sb, NULL);\n+\n+\tif (!file_exists(sm_gitdir)) {\n+\t\tif (safe_create_leading_directories_const(sm_gitdir) < 0)\n+\t\t\tdie(_(\"could not create directory '%s'\"), sm_gitdir);\n+\t\tif (clone_submodule(path, sm_gitdir, url, depth, reference, quiet))\n+\t\t\tdie(_(\"clone of '%s' into submodule path '%s' failed\"),\n+\t\t\t    url, path);\n+\t} else {\n+\t\tif (safe_create_leading_directories_const(path) < 0)\n+\t\t\tdie(_(\"could not create directory '%s'\"), path);\n+\t\tstrbuf_addf(&sb, \"%s/index\", sm_gitdir);\n+\t\tunlink_or_warn(sb.buf);\n+\t\tstrbuf_reset(&sb);\n+\t}\n+\n+\t/* Write a .git file in the submodule to redirect to the superproject. */\n+\tif (safe_create_leading_directories_const(path) < 0)\n+\t\tdie(_(\"could not create directory '%s'\"), path);\n+\n+\tif (path && *path)\n+\t\tstrbuf_addf(&sb, \"%s/.git\", path);\n+\telse\n+\t\tstrbuf_addstr(&sb, \".git\");\n+\n+\tif (safe_create_leading_directories_const(sb.buf) < 0)\n+\t\tdie(_(\"could not create leading directories of '%s'\"), sb.buf);\n+\tsubmodule_dot_git = fopen(sb.buf, \"w\");\n+\tif (!submodule_dot_git)\n+\t\tdie_errno(_(\"cannot open file '%s'\"), sb.buf);\n+\n+\tfprintf(submodule_dot_git, \"gitdir: %s\\n\",\n+\t\trelative_path(sm_gitdir, path, &rel_path));\n+\tif (fclose(submodule_dot_git))\n+\t\tdie(_(\"could not close file %s\"), sb.buf);\n+\tstrbuf_reset(&sb);\n+\tstrbuf_reset(&rel_path);\n+\n+\tcwd = xgetcwd();\n+\t/* Redirect the worktree of the submodule in the superproject's config */\n+\tif (!is_absolute_path(sm_gitdir)) {\n+\t\tstrbuf_addf(&sb, \"%s/%s\", cwd, sm_gitdir);\n+\t\tfree(sm_gitdir);\n+\t\tsm_gitdir = strbuf_detach(&sb, NULL);\n+\t}\n+\n+\tstrbuf_addf(&sb, \"%s/%s\", cwd, path);\n+\tp = git_pathdup_submodule(path, \"config\");\n+\tif (!p)\n+\t\tdie(_(\"could not get submodule directory for '%s'\"), path);\n+\tgit_config_set_in_file(p, \"core.worktree\",\n+\t\t\t       relative_path(sb.buf, sm_gitdir, &rel_path));\n+\tstrbuf_release(&sb);\n+\tstrbuf_release(&rel_path);\n+\tfree(sm_gitdir);\n+\tfree(cwd);\n+\tfree(p);\n+\treturn 0;\n+}\n \n struct cmd_struct {\n \tconst char *cmd;\n@@ -132,6 +263,7 @@ struct cmd_struct {\n static struct cmd_struct commands[] = {\n \t{\"list\", module_list},\n \t{\"name\", module_name},\n+\t{\"clone\", module_clone},\n };\n \n int cmd_submodule__helper(int argc, const char **argv, const char *prefix)\ndiff --git a/git-submodule.sh b/git-submodule.sh\nindex 2be8da2..8b0eb9a 100755\n--- a/git-submodule.sh\n+++ b/git-submodule.sh\n@@ -178,80 +178,6 @@ get_submodule_config () {\n \tprintf '%s' \"${value:-$default}\"\n }\n \n-#\n-# Clone a submodule\n-#\n-# $1 = submodule path\n-# $2 = submodule name\n-# $3 = URL to clone\n-# $4 = reference repository to reuse (empty for independent)\n-# $5 = depth argument for shallow clones (empty for deep)\n-#\n-# Prior to calling, cmd_update checks that a possibly existing\n-# path is not a git repository.\n-# Likewise, cmd_add checks that path does not exist at all,\n-# since it is the location of a new submodule.\n-#\n-module_clone()\n-{\n-\tsm_path=$1\n-\tname=$2\n-\turl=$3\n-\treference=\"$4\"\n-\tdepth=\"$5\"\n-\tquiet=\n-\tif test -n \"$GIT_QUIET\"\n-\tthen\n-\t\tquiet=-q\n-\tfi\n-\n-\tgitdir=\n-\tgitdir_base=\n-\tbase_name=$(dirname \"$name\")\n-\n-\tgitdir=$(git rev-parse --git-dir)\n-\tgitdir_base=\"$gitdir/modules/$base_name\"\n-\tgitdir=\"$gitdir/modules/$name\"\n-\n-\tif test -d \"$gitdir\"\n-\tthen\n-\t\tmkdir -p \"$sm_path\"\n-\t\trm -f \"$gitdir/index\"\n-\telse\n-\t\tmkdir -p \"$gitdir_base\"\n-\t\t(\n-\t\t\tclear_local_git_env\n-\t\t\tgit clone $quiet ${depth:+\"$depth\"} -n ${reference:+\"$reference\"} \\\n-\t\t\t\t--separate-git-dir \"$gitdir\" \"$url\" \"$sm_path\"\n-\t\t) ||\n-\t\tdie \"$(eval_gettext \"Clone of '\\$url' into submodule path '\\$sm_path' failed\")\"\n-\tfi\n-\n-\t# We already are at the root of the work tree but cd_to_toplevel will\n-\t# resolve any symlinks that might be present in $PWD\n-\ta=$(cd_to_toplevel && cd \"$gitdir\" && pwd)/\n-\tb=$(cd_to_toplevel && cd \"$sm_path\" && pwd)/\n-\t# Remove all common leading directories after a sanity check\n-\tif test \"${a#$b}\" != \"$a\" || test \"${b#$a}\" != \"$b\"; then\n-\t\tdie \"$(eval_gettext \"Gitdir '\\$a' is part of the submodule path '\\$b' or vice versa\")\"\n-\tfi\n-\twhile test \"${a%%/*}\" = \"${b%%/*}\"\n-\tdo\n-\t\ta=${a#*/}\n-\t\tb=${b#*/}\n-\tdone\n-\t# Now chop off the trailing '/'s that were added in the beginning\n-\ta=${a%/}\n-\tb=${b%/}\n-\n-\t# Turn each leading \"*/\" component into \"../\"\n-\trel=$(printf '%s\\n' \"$b\" | sed -e 's|[^/][^/]*|..|g')\n-\tprintf '%s\\n' \"gitdir: $rel/$a\" >\"$sm_path/.git\"\n-\n-\trel=$(printf '%s\\n' \"$a\" | sed -e 's|[^/][^/]*|..|g')\n-\t(clear_local_git_env; cd \"$sm_path\" && GIT_WORK_TREE=. git config core.worktree \"$rel/$b\")\n-}\n-\n isnumber()\n {\n \tn=$(($1 + 0)) 2>/dev/null && test \"$n\" = \"$1\"\n@@ -412,7 +338,7 @@ Use -f if you really want to add it.\" >&2\n \t\t\t\techo \"$(eval_gettext \"Reactivating local git directory for submodule '\\$sm_name'.\")\"\n \t\t\tfi\n \t\tfi\n-\t\tmodule_clone \"$sm_path\" \"$sm_name\" \"$realrepo\" \"$reference\" \"$depth\" || exit\n+\t\tgit submodule--helper clone ${GIT_QUIET:+--quiet} --prefix \"$wt_prefix\" --path \"$sm_path\" --name \"$sm_name\" --url \"$realrepo\" \"$reference\" \"$depth\" || exit\n \t\t(\n \t\t\tclear_local_git_env\n \t\t\tcd \"$sm_path\" &&\n@@ -774,7 +700,7 @@ Maybe you want to use 'update --init'?\")\"\n \n \t\tif ! test -d \"$sm_path\"/.git && ! test -f \"$sm_path\"/.git\n \t\tthen\n-\t\t\tmodule_clone \"$sm_path\" \"$name\" \"$url\" \"$reference\" \"$depth\" || exit\n+\t\t\tgit submodule--helper clone ${GIT_QUIET:+--quiet} --prefix \"$prefix\" --path \"$sm_path\" --name \"$name\" --url \"$url\" \"$reference\" \"$depth\" || exit\n \t\t\tcloned_modules=\"$cloned_modules;$name\"\n \t\t\tsubsha1=\n \t\telse\n-- \n2.5.0.256.g89f8063.dirty\n"}]}