{"thread":{"id":"32695","subject":"[PATCH] all: new command used for multi-repo operations","startedAt":"2013-01-22T21:10:19Z","lastAt":"2013-01-23T08:52:52Z","messageCount":11,"participants":["Lars Hjemli","Junio C Hamano","David Aguilar","Duy Nguyen"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"207526","messageId":"1358889019-4554-1-git-send-email-hjemli@gmail.com","threadId":"32695","inReplyTo":null,"subject":"[PATCH] all: new command used for multi-repo operations","fromName":"Lars Hjemli","fromEmail":"hjemli@gmail.com","sentAt":"2013-01-22T21:10:19Z","receivedAt":"2013-01-22T21:10:19Z","isPatch":true,"sender":{"key":"hjemli@gmail.com","avatar":null},"body":"When working with multiple, unrelated (or loosly related) git repos,\nthere is often a need to locate all repos with uncommitted work and\nperform some action on them (say, commit and push). Before this patch,\nsuch tasks would require manually visiting all repositories, running\n`git status` within each one and then decide what to do next.\n\nThis mundane task can now be automated by e.g. `git all --dirty status`,\nwhich will find all git repositories below the current directory (even\nnested ones), check if they are dirty (as defined by `git diff --quiet &&\ngit diff --cached --quiet`), and for each dirty repo print the path to the\nrepo and then execute `git status` within the repo.\n\nThe command also honours the option '--clean' which restricts the set of\nrepos to those which '--dirty' would skip.\n\nFinally, the command to execute within each repo is optional. If none is\ngiven, git-all will just print the path to each repo found.\n\nSigned-off-by: Lars Hjemli \n---\n Documentation/git-all.txt | 37 ++++++++++++++++\n Makefile | 1 +\n builtin.h | 1 +\n builtin/all.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++\n command-list.txt | 1 +\n git.c | 1 +\n t/t0064-all.sh | 42 +++++++++++++++++++\n 7 files changed, 188 insertions(+)\n create mode 100644 Documentation/git-all.txt\n create mode 100644 builtin/all.c\n create mode 100755 t/t0064-all.sh\n\ndiff --git a/Documentation/git-all.txt b/Documentation/git-all.txt\nnew file mode 100644\nindex 0000000..b25f23c\n--- /dev/null\n+++ b/Documentation/git-all.txt\n@@ -0,0 +1,37 @@\n+git-all(1)\n+==========\n+\n+NAME\n+----\n+git-all - Execute a git command in multiple repositories\n+\n+SYNOPSIS\n+--------\n+[verse]\n+'git all' [--dirty|--clean] [command]\n+\n+DESCRIPTION\n+-----------\n+The git-all command is used to locate all git repositoris within the\n+current directory tree, and optionally execute a git command in each\n+of the found repos.\n+\n+OPTIONS\n+-------\n+-c::\n+--clean::\n+\tOnly include repositories with a clean worktree.\n+\n+-d::\n+--dirty::\n+\tOnly include repositories with a dirty worktree.\n+\n+NOTES\n+-----\n+\n+For the purpose of `git-all`, a dirty worktree is defined as a worktree\n+with uncommitted changes.\n+\n+GIT\n+---\n+Part of the linkgit:git[1] suite\ndiff --git a/Makefile b/Makefile\nindex 1b30d7b..8bf0583 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -840,6 +840,7 @@ LIB_OBJS += xdiff-interface.o\n LIB_OBJS += zlib.o\n \n BUILTIN_OBJS += builtin/add.o\n+BUILTIN_OBJS += builtin/all.o\n BUILTIN_OBJS += builtin/annotate.o\n BUILTIN_OBJS += builtin/apply.o\n BUILTIN_OBJS += builtin/archive.o\ndiff --git a/builtin.h b/builtin.h\nindex 7e7bbd6..438c265 100644\n--- a/builtin.h\n+++ b/builtin.h\n@@ -41,6 +41,7 @@ void finish_copy_notes_for_rewrite(struct notes_rewrite_cfg *c);\n extern int textconv_object(const char *path, unsigned mode, const unsigned char *sha1, int sha1_valid, char **buf, unsigned long *buf_size);\n \n extern int cmd_add(int argc, const char **argv, const char *prefix);\n+extern int cmd_all(int argc, const char **argv, const char *prefix);\n extern int cmd_annotate(int argc, const char **argv, const char *prefix);\n extern int cmd_apply(int argc, const char **argv, const char *prefix);\n extern int cmd_archive(int argc, const char **argv, const char *prefix);\ndiff --git a/builtin/all.c b/builtin/all.c\nnew file mode 100644\nindex 0000000..ee9270d\n--- /dev/null\n+++ b/builtin/all.c\n@@ -0,0 +1,105 @@\n+/*\n+ * \"git all\" builtin command.\n+ *\n+ * Copyright (c) 2013 Lars Hjemli \n+ */\n+#include \"cache.h\"\n+#include \"color.h\"\n+#include \"builtin.h\"\n+#include \"run-command.h\"\n+#include \"parse-options.h\"\n+\n+static int only_dirty;\n+static int only_clean;\n+char root[PATH_MAX];\n+\n+static const char * const builtin_all_usage[] = {\n+\tN_(\"git all [options] [cmd]\"),\n+\tNULL\n+};\n+\n+static struct option builtin_all_options[] = {\n+\tOPT_BOOLEAN('c', \"clean\", &only_clean, N_(\"only show clean repositories\")),\n+\tOPT_BOOLEAN('d', \"dirty\", &only_dirty, N_(\"only show dirty repositories\")),\n+\tOPT_END(),\n+};\n+\n+static int is_dirty()\n+{\n+\tconst char *diffidx[] = {\"diff\", \"--quiet\", \"--cached\", NULL};\n+\tconst char *diffwd[] = {\"diff\", \"--quiet\", NULL};\n+\n+\tif (run_command_v_opt(diffidx, RUN_GIT_CMD) != 0)\n+\t\treturn 1;\n+\tif (run_command_v_opt(diffwd, RUN_GIT_CMD) != 0)\n+\t\treturn 1;\n+\treturn 0;\n+}\n+\n+static void handle_repo(char *path, const char **argv)\n+{\n+\tint dirty;\n+\n+\tif (path[0] == '.' && path[1] == '/')\n+\t\tpath += 2;\n+\tif (only_dirty || only_clean) {\n+\t\tdirty = is_dirty();\n+\t\tif ((dirty && only_clean) ||\n+\t\t (!dirty && only_dirty))\n+\t\t\treturn;\n+\t}\n+\tif (*argv) {\n+\t\tcolor_fprintf_ln(stdout, GIT_COLOR_YELLOW, \"[%s]\", path);\n+\t\trun_command_v_opt(argv, RUN_GIT_CMD);\n+\t} else\n+\t\tprintf(\"%s\\n\", path);\n+}\n+\n+static int walk(struct strbuf *path, int argc, const char **argv)\n+{\n+\tDIR *dir;\n+\tstruct dirent *ent;\n+\tsize_t len;\n+\n+\tdir = opendir(path->buf);\n+\tif (!dir)\n+\t\treturn errno;\n+\tstrbuf_addstr(path, \"/\");\n+\tlen = path->len;\n+\twhile ((ent = readdir(dir))) {\n+\t\tif (!strcmp(ent->d_name, \".\") || !strcmp(ent->d_name, \"..\"))\n+\t\t\tcontinue;\n+\t\tif (!strcmp(ent->d_name, \".git\")) {\n+\t\t\tstrbuf_setlen(path, len - 1);\n+\t\t\tchdir(path->buf);\n+\t\t\thandle_repo(path->buf, argv);\n+\t\t\tchdir(root);\n+\t\t\tstrbuf_addstr(path, \"/\");\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (ent->d_type != DT_DIR)\n+\t\t\tcontinue;\n+\t\tstrbuf_setlen(path, len);\n+\t\tstrbuf_addstr(path, ent->d_name);\n+\t\twalk(path, argc, argv);\n+\t}\n+\tclosedir(dir);\n+\treturn 0;\n+}\n+\n+int cmd_all(int argc, const char **argv, const char *prefix)\n+{\n+\tstruct strbuf path = STRBUF_INIT;\n+\n+\tif (!getcwd(root, sizeof(root)))\n+\t\treturn 1;\n+\n+\targc = parse_options(argc, argv, prefix, builtin_all_options,\n+\t\t\t builtin_all_usage, PARSE_OPT_STOP_AT_NON_OPTION);\n+\n+\tunsetenv(GIT_DIR_ENVIRONMENT);\n+\tunsetenv(GIT_WORK_TREE_ENVIRONMENT);\n+\n+\tstrbuf_addstr(&path, \".\");\n+\treturn walk(&path, argc, argv);\n+}\ndiff --git a/command-list.txt b/command-list.txt\nindex 7e8cfec..f955895 100644\n--- a/command-list.txt\n+++ b/command-list.txt\n@@ -1,6 +1,7 @@\n # List of known git commands.\n # command name\t\t\t\tcategory [deprecated] [common]\n git-add mainporcelain common\n+git-all mainporcelain\n git-am mainporcelain\n git-annotate ancillaryinterrogators\n git-apply plumbingmanipulators\ndiff --git a/git.c b/git.c\nindex ed66c66..53fd963 100644\n--- a/git.c\n+++ b/git.c\n@@ -304,6 +304,7 @@ static void handle_internal_command(int argc, const char **argv)\n \tconst char *cmd = argv[0];\n \tstatic struct cmd_struct commands[] = {\n \t\t{ \"add\", cmd_add, RUN_SETUP | NEED_WORK_TREE },\n+\t\t{ \"all\", cmd_all },\n \t\t{ \"annotate\", cmd_annotate, RUN_SETUP },\n \t\t{ \"apply\", cmd_apply, RUN_SETUP_GENTLY },\n \t\t{ \"archive\", cmd_archive },\ndiff --git a/t/t0064-all.sh b/t/t0064-all.sh\nnew file mode 100755\nindex 0000000..932e374\n--- /dev/null\n+++ b/t/t0064-all.sh\n@@ -0,0 +1,42 @@\n+#!/bin/sh\n+#\n+# Copyright (c) 2013 Lars Hjemli\n+#\n+\n+test_description='Test the git-all command'\n+\n+. ./test-lib.sh\n+\n+test_expect_success \"setup\" '\n+\ttest_create_repo clean &&\n+\t(cd clean && test_commit foo) &&\n+\ttest_create_repo dirty-wt &&\n+\t(cd dirty-wt && test_commit foo && rm foo.t) &&\n+\ttest_create_repo dirty-idx &&\n+\t(cd dirty-idx && test_commit foo && git rm foo.t)\n+'\n+\n+test_expect_success \"without flags, all repos are included\" '\n+\techo \".\" >expect &&\n+\techo \"clean\" >>expect &&\n+\techo \"dirty-idx\" >>expect &&\n+\techo \"dirty-wt\" >>expect &&\n+\tgit all | sort >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success \"--dirty only includes dirty repos\" '\n+\techo \"dirty-idx\" >expect &&\n+\techo \"dirty-wt\" >>expect &&\n+\tgit all --dirty | sort >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_expect_success \"--clean only includes clean repos\" '\n+\techo \".\" >expect &&\n+\techo \"clean\" >>expect &&\n+\tgit all --clean | sort >actual &&\n+\ttest_cmp expect actual\n+'\n+\n+test_done\n-- \n1.8.1.1.296.g725455c\n"},{"id":"207529","messageId":"7vmww0nahd.fsf@alter.siamese.dyndns.org","threadId":"32695","inReplyTo":"1358889019-4554-1-git-send-email-hjemli@gmail.com","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-22T22:01:18Z","receivedAt":"2013-01-22T22:01:18Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Hjemli writes:\n\n> diff --git a/builtin/all.c b/builtin/all.c\n> new file mode 100644\n> index 0000000..ee9270d\n> --- /dev/null\n> +++ b/builtin/all.c\n> @@ -0,0 +1,105 @@\n> +/*\n> + * \"git all\" builtin command.\n> + *\n> + * Copyright (c) 2013 Lars Hjemli \n> + */\n> +#include \"cache.h\"\n> +#include \"color.h\"\n> +#include \"builtin.h\"\n> +#include \"run-command.h\"\n> +#include \"parse-options.h\"\n> +\n> +static int only_dirty;\n> +static int only_clean;\n> +char root[PATH_MAX];\n> +\n> +static const char * const builtin_all_usage[] = {\n> +\tN_(\"git all [options] [cmd]\"),\n> +\tNULL\n> +};\n> +\n> +static struct option builtin_all_options[] = {\n> +\tOPT_BOOLEAN('c', \"clean\", &only_clean, N_(\"only show clean repositories\")),\n> +\tOPT_BOOLEAN('d', \"dirty\", &only_dirty, N_(\"only show dirty repositories\")),\n> +\tOPT_END(),\n> +};\n\nShouldn't this be more like OPT_SET_INT() on a same variable that is\ninitialized to \"all\"? Alternatively you could validate the input\nand die when both are given.\n\n> +int cmd_all(int argc, const char **argv, const char *prefix)\n> +{\n> +\tstruct strbuf path = STRBUF_INIT;\n> +\n> +\tif (!getcwd(root, sizeof(root)))\n> +\t\treturn 1;\n> +\n> +\targc = parse_options(argc, argv, prefix, builtin_all_options,\n> +\t\t\t builtin_all_usage, PARSE_OPT_STOP_AT_NON_OPTION);\n> +\n> +\tunsetenv(GIT_DIR_ENVIRONMENT);\n> +\tunsetenv(GIT_WORK_TREE_ENVIRONMENT);\n\nDon't you need to clear other variables whose uses are closely tied\nto a single repository, like GIT_INDEX_FILE, etc.?\n\nI suspect that explicitly exporting GIT_DIR and GIT_WORK_TREE (and\nnothing else) in handle_repo() to the location you discovered before\nyou run the per-repository command via run_command_v_opt(), might be\na better alternative. The user could be sharing objects in all\nrepositories by permanently setting GIT_OBJECT_DIRECTORY to a single\nplace.\n\n> diff --git a/command-list.txt b/command-list.txt\n> index 7e8cfec..f955895 100644\n> --- a/command-list.txt\n> +++ b/command-list.txt\n> @@ -1,6 +1,7 @@\n> # List of known git commands.\n> # command name\t\t\t\tcategory [deprecated] [common]\n> git-add mainporcelain common\n> +git-all mainporcelain\n> git-am mainporcelain\n> git-annotate ancillaryinterrogators\n> git-apply plumbingmanipulators\n\nI am not very interested in this topic in the first place, but this\ndoes not (at least not yet) sound like a main Porcelain to me.\n\n\"all\" may be a word other people may want to use to call collections\nof things other than \"Git repositories\", and that use may turn out\nto be more useful in general. A name that makes it clear that this\nis about \"repositories\", i.e. along the lines of \"git for-each-repo\"\nor something, would be a better name that does not squat on such a\nshort and sweet name.\n"},{"id":"207536","messageId":"CAFXTnz62H24G9Zp5EgG47MyNdDz0KddX2n34G_1-RV_yP-ELkA@mail.gmail.com","threadId":"32695","inReplyTo":"7vmww0nahd.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"Lars Hjemli","fromEmail":"hjemli@gmail.com","sentAt":"2013-01-22T23:35:12Z","receivedAt":"2013-01-22T23:35:12Z","isPatch":true,"sender":{"key":"hjemli@gmail.com","avatar":null},"body":"On Tue, Jan 22, 2013 at 11:01 PM, Junio C Hamano wrote:\n> Lars Hjemli writes:\n>\n>> +static struct option builtin_all_options[] = {\n>> + OPT_BOOLEAN('c', \"clean\", &only_clean, N_(\"only show clean repositories\")),\n>> + OPT_BOOLEAN('d', \"dirty\", &only_dirty, N_(\"only show dirty repositories\")),\n>> + OPT_END(),\n>> +};\n>\n> Shouldn't this be more like OPT_SET_INT() on a same variable that is\n> initialized to \"all\"? Alternatively you could validate the input\n> and die when both are given.\n\nOPT_SET_INT() seems appropriate, will fix.\n\n>\n>> +int cmd_all(int argc, const char **argv, const char *prefix)\n>> +{\n>> + struct strbuf path = STRBUF_INIT;\n>> +\n>> + if (!getcwd(root, sizeof(root)))\n>> + return 1;\n>> +\n>> + argc = parse_options(argc, argv, prefix, builtin_all_options,\n>> + builtin_all_usage, PARSE_OPT_STOP_AT_NON_OPTION);\n>> +\n>> + unsetenv(GIT_DIR_ENVIRONMENT);\n>> + unsetenv(GIT_WORK_TREE_ENVIRONMENT);\n>\n> Don't you need to clear other variables whose uses are closely tied\n> to a single repository, like GIT_INDEX_FILE, etc.?\n>\n> I suspect that explicitly exporting GIT_DIR and GIT_WORK_TREE (and\n> nothing else) in handle_repo() to the location you discovered before\n> you run the per-repository command via run_command_v_opt(), might be\n> a better alternative. The user could be sharing objects in all\n> repositories by permanently setting GIT_OBJECT_DIRECTORY to a single\n> place.\n>\n\nThis sounds like a nice plan, I'll test it and send an updated patch.\n\n\n>> diff --git a/command-list.txt b/command-list.txt\n>> index 7e8cfec..f955895 100644\n>> --- a/command-list.txt\n>> +++ b/command-list.txt\n>> @@ -1,6 +1,7 @@\n>> # List of known git commands.\n>> # command name category [deprecated] [common]\n>> git-add mainporcelain common\n>> +git-all mainporcelain\n>> git-am mainporcelain\n>> git-annotate ancillaryinterrogators\n>> git-apply plumbingmanipulators\n>\n> I am not very interested in this topic in the first place, but this\n> does not (at least not yet) sound like a main Porcelain to me.\n\nThere doesn't seem to be a better category, but I'm open for suggestions.\n\n>\n> \"all\" may be a word other people may want to use to call collections\n> of things other than \"Git repositories\", and that use may turn out\n> to be more useful in general. A name that makes it clear that this\n> is about \"repositories\", i.e. along the lines of \"git for-each-repo\"\n> or something, would be a better name that does not squat on such a\n> short and sweet name.\n>\n\nIn principle I agree with your reasoning on this, but in practice I\nfail to see what other kind of things `git all` could naturally refer\nto. Also, having a short and sweet way to perform the tasks\nimplemented by this patch was my main motivation for writing it [1],\nhence `git for-each-repo` isn't as compelling (too much typing).\nThere's always aliases, but I'd prefer it if future git supported `git\nall` by default.\n\n-- \nlarsh\n\n[1] Originally as a shell script, used at $WORK for ~2 years\n"},{"id":"207540","messageId":"7vk3r4lprs.fsf@alter.siamese.dyndns.org","threadId":"32695","inReplyTo":"CAFXTnz62H24G9Zp5EgG47MyNdDz0KddX2n34G_1-RV_yP-ELkA@mail.gmail.com","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-23T00:13:59Z","receivedAt":"2013-01-23T00:13:59Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Hjemli writes:\n\n> In principle I agree with your reasoning on this, but in practice I\n> fail to see what other kind of things `git all` could naturally refer\n> to.\n\nFor example, the designers of 'for-each-ref' could have called it\n\"git all\", as it is to iterate over all refs.\n"},{"id":"207542","messageId":"CAJDDKr6exXh14m08HTihxREjSFgyPT0bN1cF8eUryXJHOgFL1A@mail.gmail.com","threadId":"32695","inReplyTo":"1358889019-4554-1-git-send-email-hjemli@gmail.com","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"David Aguilar","fromEmail":"davvid@gmail.com","sentAt":"2013-01-23T00:43:07Z","receivedAt":"2013-01-23T00:43:07Z","isPatch":true,"sender":{"key":"davvid@gmail.com","avatar":"https://avatars.githubusercontent.com/u/13196?v=4"},"body":"On Tue, Jan 22, 2013 at 1:10 PM, Lars Hjemli wrote:\n> When working with multiple, unrelated (or loosly related) git repos,\n> there is often a need to locate all repos with uncommitted work and\n> perform some action on them (say, commit and push). Before this patch,\n> such tasks would require manually visiting all repositories, running\n> `git status` within each one and then decide what to do next.\n>\n> This mundane task can now be automated by e.g. `git all --dirty status`,\n> which will find all git repositories below the current directory (even\n> nested ones), check if they are dirty (as defined by `git diff --quiet &&\n> git diff --cached --quiet`), and for each dirty repo print the path to the\n> repo and then execute `git status` within the repo.\n>\n> The command also honours the option '--clean' which restricts the set of\n> repos to those which '--dirty' would skip.\n>\n> Finally, the command to execute within each repo is optional. If none is\n> given, git-all will just print the path to each repo found.\n>\n> Signed-off-by: Lars Hjemli \n> ---\n> Documentation/git-all.txt | 37 ++++++++++++++++\n> Makefile | 1 +\n> builtin.h | 1 +\n> builtin/all.c | 105 ++++++++++++++++++++++++++++++++++++++++++++++\n> command-list.txt | 1 +\n> git.c | 1 +\n> t/t0064-all.sh | 42 +++++++++++++++++++\n> 7 files changed, 188 insertions(+)\n> create mode 100644 Documentation/git-all.txt\n> create mode 100644 builtin/all.c\n> create mode 100755 t/t0064-all.sh\n>\n> diff --git a/Documentation/git-all.txt b/Documentation/git-all.txt\n> new file mode 100644\n> index 0000000..b25f23c\n> --- /dev/null\n> +++ b/Documentation/git-all.txt\n> @@ -0,0 +1,37 @@\n> +git-all(1)\n> +==========\n> +\n> +NAME\n> +----\n> +git-all - Execute a git command in multiple repositories\n> +\n> +SYNOPSIS\n> +--------\n> +[verse]\n> +'git all' [--dirty|--clean] [command]\n> +\n> +DESCRIPTION\n> +-----------\n> +The git-all command is used to locate all git repositoris within the\n> +current directory tree, and optionally execute a git command in each\n> +of the found repos.\n> +\n> +OPTIONS\n> +-------\n> +-c::\n> +--clean::\n> + Only include repositories with a clean worktree.\n> +\n> +-d::\n> +--dirty::\n> + Only include repositories with a dirty worktree.\n> +\n> +NOTES\n> +-----\n> +\n> +For the purpose of `git-all`, a dirty worktree is defined as a worktree\n> +with uncommitted changes.\n> +\n> +GIT\n> +---\n> +Part of the linkgit:git[1] suite\n> diff --git a/Makefile b/Makefile\n> index 1b30d7b..8bf0583 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -840,6 +840,7 @@ LIB_OBJS += xdiff-interface.o\n> LIB_OBJS += zlib.o\n>\n> BUILTIN_OBJS += builtin/add.o\n> +BUILTIN_OBJS += builtin/all.o\n> BUILTIN_OBJS += builtin/annotate.o\n> BUILTIN_OBJS += builtin/apply.o\n> BUILTIN_OBJS += builtin/archive.o\n> diff --git a/builtin.h b/builtin.h\n> index 7e7bbd6..438c265 100644\n> --- a/builtin.h\n> +++ b/builtin.h\n> @@ -41,6 +41,7 @@ void finish_copy_notes_for_rewrite(struct notes_rewrite_cfg *c);\n> extern int textconv_object(const char *path, unsigned mode, const unsigned char *sha1, int sha1_valid, char **buf, unsigned long *buf_size);\n>\n> extern int cmd_add(int argc, const char **argv, const char *prefix);\n> +extern int cmd_all(int argc, const char **argv, const char *prefix);\n> extern int cmd_annotate(int argc, const char **argv, const char *prefix);\n> extern int cmd_apply(int argc, const char **argv, const char *prefix);\n> extern int cmd_archive(int argc, const char **argv, const char *prefix);\n> diff --git a/builtin/all.c b/builtin/all.c\n> new file mode 100644\n> index 0000000..ee9270d\n> --- /dev/null\n> +++ b/builtin/all.c\n> @@ -0,0 +1,105 @@\n> +/*\n> + * \"git all\" builtin command.\n> + *\n> + * Copyright (c) 2013 Lars Hjemli \n> + */\n> +#include \"cache.h\"\n> +#include \"color.h\"\n> +#include \"builtin.h\"\n> +#include \"run-command.h\"\n> +#include \"parse-options.h\"\n> +\n> +static int only_dirty;\n> +static int only_clean;\n> +char root[PATH_MAX];\n> +\n> +static const char * const builtin_all_usage[] = {\n> + N_(\"git all [options] [cmd]\"),\n> + NULL\n> +};\n> +\n> +static struct option builtin_all_options[] = {\n> + OPT_BOOLEAN('c', \"clean\", &only_clean, N_(\"only show clean repositories\")),\n> + OPT_BOOLEAN('d', \"dirty\", &only_dirty, N_(\"only show dirty repositories\")),\n> + OPT_END(),\n> +};\n> +\n> +static int is_dirty()\n> +{\n> + const char *diffidx[] = {\"diff\", \"--quiet\", \"--cached\", NULL};\n> + const char *diffwd[] = {\"diff\", \"--quiet\", NULL};\n> +\n> + if (run_command_v_opt(diffidx, RUN_GIT_CMD) != 0)\n> + return 1;\n> + if (run_command_v_opt(diffwd, RUN_GIT_CMD) != 0)\n> + return 1;\n> + return 0;\n> +}\n> +\n> +static void handle_repo(char *path, const char **argv)\n> +{\n> + int dirty;\n> +\n> + if (path[0] == '.' && path[1] == '/')\n> + path += 2;\n> + if (only_dirty || only_clean) {\n> + dirty = is_dirty();\n> + if ((dirty && only_clean) ||\n> + (!dirty && only_dirty))\n> + return;\n> + }\n> + if (*argv) {\n> + color_fprintf_ln(stdout, GIT_COLOR_YELLOW, \"[%s]\", path);\n> + run_command_v_opt(argv, RUN_GIT_CMD);\n> + } else\n> + printf(\"%s\\n\", path);\n> +}\n> +\n> +static int walk(struct strbuf *path, int argc, const char **argv)\n> +{\n> + DIR *dir;\n> + struct dirent *ent;\n> + size_t len;\n> +\n> + dir = opendir(path->buf);\n> + if (!dir)\n> + return errno;\n> + strbuf_addstr(path, \"/\");\n> + len = path->len;\n> + while ((ent = readdir(dir))) {\n> + if (!strcmp(ent->d_name, \".\") || !strcmp(ent->d_name, \"..\"))\n> + continue;\n> + if (!strcmp(ent->d_name, \".git\")) {\n> + strbuf_setlen(path, len - 1);\n> + chdir(path->buf);\n> + handle_repo(path->buf, argv);\n> + chdir(root);\n> + strbuf_addstr(path, \"/\");\n> + continue;\n> + }\n\nDoes this section above properly handle .git files (where .git is a\nfile, not a directory)? I wonder whether the check should be tighter,\ne.g. something closer to what's done in setup.c:is_git_repository().\n\nThe name of this function leads me to believe that this walks\neverything below the current directory looking for Git repos. How\ndeep does it walk? Does this handle nested repositories, e.g.\nfoo/bar/ and foo/bar/baz/ when you are inside foo/ ? After re-reading\nthe documentation I am led to believe that it only walks one level\ndeep.\n\nI did not notice a test for nested repos which is what sparked my\ncuriosity. ;-) If we do not expect to handle them in the first\nversion then we should have a test to ensure the expected behavior.\nIt would also be nice to see a test with a .git file.\n\nI do wonder what the end user experience is with this command when\nused alongside other Git aggregate commands such as \"repo\" or \"git\nsubmodule\". This command is basically \"git submodule foreach ...\"\nwithout needing to buy into the whole submodule thing. This is an\nargument for naming it something like \"git foreach-repo\" since it\nwould be named more closely to the \"foreach\" submodule command.\n\nWhile \"all\" is less to type than \"foreach-repo\", most of the extra\nwork can be eliminated by installing the wonderful git completion\nscripts for bash/zsh.\n\nAll that said, I have very real use cases for this command. Thanks\nfor writing it.\n\n> + if (ent->d_type != DT_DIR)\n> + continue;\n> + strbuf_setlen(path, len);\n> + strbuf_addstr(path, ent->d_name);\n> + walk(path, argc, argv);\n> + }\n> + closedir(dir);\n> + return 0;\n> +}\n> +\n> +int cmd_all(int argc, const char **argv, const char *prefix)\n> +{\n> + struct strbuf path = STRBUF_INIT;\n> +\n> + if (!getcwd(root, sizeof(root)))\n> + return 1;\n> +\n> + argc = parse_options(argc, argv, prefix, builtin_all_options,\n> + builtin_all_usage, PARSE_OPT_STOP_AT_NON_OPTION);\n> +\n> + unsetenv(GIT_DIR_ENVIRONMENT);\n> + unsetenv(GIT_WORK_TREE_ENVIRONMENT);\n> +\n> + strbuf_addstr(&path, \".\");\n> + return walk(&path, argc, argv);\n> +}\n> diff --git a/command-list.txt b/command-list.txt\n> index 7e8cfec..f955895 100644\n> --- a/command-list.txt\n> +++ b/command-list.txt\n> @@ -1,6 +1,7 @@\n> # List of known git commands.\n> # command name category [deprecated] [common]\n> git-add mainporcelain common\n> +git-all mainporcelain\n> git-am mainporcelain\n> git-annotate ancillaryinterrogators\n> git-apply plumbingmanipulators\n> diff --git a/git.c b/git.c\n> index ed66c66..53fd963 100644\n> --- a/git.c\n> +++ b/git.c\n> @@ -304,6 +304,7 @@ static void handle_internal_command(int argc, const char **argv)\n> const char *cmd = argv[0];\n> static struct cmd_struct commands[] = {\n> { \"add\", cmd_add, RUN_SETUP | NEED_WORK_TREE },\n> + { \"all\", cmd_all },\n> { \"annotate\", cmd_annotate, RUN_SETUP },\n> { \"apply\", cmd_apply, RUN_SETUP_GENTLY },\n> { \"archive\", cmd_archive },\n> diff --git a/t/t0064-all.sh b/t/t0064-all.sh\n> new file mode 100755\n> index 0000000..932e374\n> --- /dev/null\n> +++ b/t/t0064-all.sh\n> @@ -0,0 +1,42 @@\n> +#!/bin/sh\n> +#\n> +# Copyright (c) 2013 Lars Hjemli\n> +#\n> +\n> +test_description='Test the git-all command'\n> +\n> +. ./test-lib.sh\n> +\n> +test_expect_success \"setup\" '\n> + test_create_repo clean &&\n> + (cd clean && test_commit foo) &&\n> + test_create_repo dirty-wt &&\n> + (cd dirty-wt && test_commit foo && rm foo.t) &&\n> + test_create_repo dirty-idx &&\n> + (cd dirty-idx && test_commit foo && git rm foo.t)\n> +'\n> +\n> +test_expect_success \"without flags, all repos are included\" '\n> + echo \".\" >expect &&\n> + echo \"clean\" >>expect &&\n> + echo \"dirty-idx\" >>expect &&\n> + echo \"dirty-wt\" >>expect &&\n> + git all | sort >actual &&\n> + test_cmp expect actual\n> +'\n> +\n> +test_expect_success \"--dirty only includes dirty repos\" '\n> + echo \"dirty-idx\" >expect &&\n> + echo \"dirty-wt\" >>expect &&\n> + git all --dirty | sort >actual &&\n> + test_cmp expect actual\n> +'\n> +\n> +test_expect_success \"--clean only includes clean repos\" '\n> + echo \".\" >expect &&\n> + echo \"clean\" >>expect &&\n> + git all --clean | sort >actual &&\n> + test_cmp expect actual\n> +'\n> +\n> +test_done\n> --\n> 1.8.1.1.296.g725455c\n>\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at http://vger.kernel.org/majordomo-info.html\n\n\n\n-- \nDavid\n"},{"id":"207565","messageId":"7vy5fkiek0.fsf@alter.siamese.dyndns.org","threadId":"32695","inReplyTo":"CAJDDKr6exXh14m08HTihxREjSFgyPT0bN1cF8eUryXJHOgFL1A@mail.gmail.com","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-23T06:44:31Z","receivedAt":"2013-01-23T06:44:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"David Aguilar writes:\n\n>> +static int walk(struct strbuf *path, int argc, const char **argv)\n>> +{\n>> + DIR *dir;\n>> + struct dirent *ent;\n>> + size_t len;\n>> +\n>> + dir = opendir(path->buf);\n>> + if (!dir)\n>> + return errno;\n>> + strbuf_addstr(path, \"/\");\n>> + len = path->len;\n>> + while ((ent = readdir(dir))) {\n>> + if (!strcmp(ent->d_name, \".\") || !strcmp(ent->d_name, \"..\"))\n>> + continue;\n>> + if (!strcmp(ent->d_name, \".git\")) {\n>> + strbuf_setlen(path, len - 1);\n>> + chdir(path->buf);\n>> + handle_repo(path->buf, argv);\n>> + chdir(root);\n>> + strbuf_addstr(path, \"/\");\n>> + continue;\n>> + }\n>\n> Does this section above properly handle .git files (where .git is a\n> file, not a directory)?\n\nThis scans a directory $D to ask \"is there '.git' in you?\" and if\nthe answer is \"yes\", then hands $D (not \"$D/.git\") to handle_repo().\nThat logic will not miss a gitfile that points at the real $GIT_DIR\nelsewhere.\n\nThere is a recursive call to walk() later in the same loop when the\nfound entry ent turns out to be a directory, and \"$D/\" + ent->d_name\nis given to this function.\n\nBut I do not think the loop structure of this function is right. If\n$D has \".git\" in it, should it even try to feed other subdirectories\nof $D (say \"$D/a\") to itself in recursion to see if $D/a/.git exists?\n\nI think it should be more like\n\n\twalk(struct strbuf *path)\n {\n\t\tsize_t dirlen = path->len;\n\t\tint has_git;\n\n\t\tstrbuf_addstr(path, \"/.git\");\n\t\thas_git = !lstat(path->buf);\n\t\tstrbuf_setlen(path, dirlen);\n\n if (has_git) {\n handle_repo(path->buf);\n\t\t\treturn;\n\t\t}\n dir = opendir(path->buf);\n\t\twhile ((ent = readdir(dir))) {\n\t\t\t... skip . and .. ...\n\t\t\tstrbuf_addstr(path, ent->d_name);\n\t\t\twalk(path);\n\t\t\tstrbuf_setlen(path, dirlen);\n }\n\t}\n\nThe determination of has_git can be a bit fancier than a simple\n!lstat() as you mentioned.\n"},{"id":"207566","messageId":"7vtxq8ie63.fsf@alter.siamese.dyndns.org","threadId":"32695","inReplyTo":"1358889019-4554-1-git-send-email-hjemli@gmail.com","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2013-01-23T06:52:52Z","receivedAt":"2013-01-23T06:52:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Lars Hjemli writes:\n\n> +static struct option builtin_all_options[] = {\n> +\tOPT_BOOLEAN('c', \"clean\", &only_clean, N_(\"only show clean repositories\")),\n> +\tOPT_BOOLEAN('d', \"dirty\", &only_dirty, N_(\"only show dirty repositories\")),\n> +\tOPT_END(),\n> +};\n\nIf you were to go in the OPT_SET_INT route, that would give users\nthe usual \"last one wins\" semantics, e.g.\n\n\t$ git for-each-repo --clean --dirty\n\nwill look for only dirty repositories. For completeness, we would\nprobably want \"all\" to defeat either of them, i.e.\n\n\t$ git for-each-repo --clean --all\n\n> +static int walk(struct strbuf *path, int argc, const char **argv)\n> +{\n> +\tDIR *dir;\n> +\tstruct dirent *ent;\n> +\tsize_t len;\n> +\n> +\tdir = opendir(path->buf);\n> +\tif (!dir)\n> +\t\treturn errno;\n> +\tstrbuf_addstr(path, \"/\");\n> +\tlen = path->len;\n> +\twhile ((ent = readdir(dir))) {\n> +\t\tif (!strcmp(ent->d_name, \".\") || !strcmp(ent->d_name, \"..\"))\n> +\t\t\tcontinue;\n> +\t\tif (!strcmp(ent->d_name, \".git\")) {\n\nThis only looks for the top of working tree. Have you considered if\nthis \"iterate over directories and list git repositories in them\"\nmay be useful for collection of bare repositories, and if it is, how\nto go about implementing the discovery process?\n\n> +\t\tif (ent->d_type != DT_DIR)\n> +\t\t\tcontinue;\n\nI think this is wrong.\n\nOn platforms that need a NO_D_TYPE_IN_DIRENT build, your compilation\nmay fail here (you would need to lstat() it yourself). See how\ndir.c does this without ugly #ifdef's in the code, especially around\nthe use of get_dtype() and DTYPE() macro.\n"},{"id":"207570","messageId":"CAFXTnz7O_nGu9AAFs_u0o+Ne4WtQT03QP6-fix7YGwpoSY+zWw@mail.gmail.com","threadId":"32695","inReplyTo":"7vy5fkiek0.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"Lars Hjemli","fromEmail":"hjemli@gmail.com","sentAt":"2013-01-23T07:39:45Z","receivedAt":"2013-01-23T07:39:45Z","isPatch":true,"sender":{"key":"hjemli@gmail.com","avatar":null},"body":"On Wed, Jan 23, 2013 at 7:44 AM, Junio C Hamano wrote:\n> But I do not think the loop structure of this function is right. If\n> $D has \".git\" in it, should it even try to feed other subdirectories\n> of $D (say \"$D/a\") to itself in recursion to see if $D/a/.git exists?\n\nYes, this is needed to meet one of the goals for git-all: to be an\nalternative to git-submodule when submodules doesn't fit (the other\ngoal is to manage a collection of unrelated (not nested) repos).\n\n-- \nlarsh\n"},{"id":"207572","messageId":"CACsJy8CTJ--u+KCYmK-2+K3NpEn72xnYwh_Pb+3Wn7nEtL1gqQ@mail.gmail.com","threadId":"32695","inReplyTo":"1358889019-4554-1-git-send-email-hjemli@gmail.com","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"Duy Nguyen","fromEmail":"pclouds@gmail.com","sentAt":"2013-01-23T08:39:18Z","receivedAt":"2013-01-23T08:39:18Z","isPatch":true,"sender":{"key":"pclouds@gmail.com","avatar":"https://avatars.githubusercontent.com/u/720?v=4"},"body":"On Wed, Jan 23, 2013 at 4:10 AM, Lars Hjemli wrote:\n> When working with multiple, unrelated (or loosly related) git repos,\n> there is often a need to locate all repos with uncommitted work and\n> perform some action on them (say, commit and push). Before this patch,\n> such tasks would require manually visiting all repositories, running\n> `git status` within each one and then decide what to do next.\n>\n> This mundane task can now be automated by e.g. `git all --dirty status`,\n> which will find all git repositories below the current directory (even\n> nested ones), check if they are dirty (as defined by `git diff --quiet &&\n> git diff --cached --quiet`), and for each dirty repo print the path to the\n> repo and then execute `git status` within the repo.\n\nI think it should leave out the execute part. The command, say\nls-repo, lists repositories in specified state. The execute part could\nbe easily done by\n\nxargs -I{} git --git-dir={} status blah\n\nI haven't thought it through. I know xargs does not support chdir'ing\ninto a repo, so maybe a new git-xargs could be introduced for that.\nBut there's still a problem with git-xargs (or git-all), printed paths\nare relative to the subrepos, not where the git-all/xargs command is\nexecuted. This could be confusing. Personally I'd like to do it\nwithout chdir. That is\n\nxargs -I{} git --git-dir={}/.git --work-tree={} status blah\n\nshould print paths relative to current working directory even if cwd\nis outside the repo.\n-- \nDuy\n"},{"id":"207573","messageId":"CAFXTnz6_8uUOCiR1MFJuYKjbtXKK4=8g-saz5Hf_KRF3EdPwEw@mail.gmail.com","threadId":"32695","inReplyTo":"CACsJy8CTJ--u+KCYmK-2+K3NpEn72xnYwh_Pb+3Wn7nEtL1gqQ@mail.gmail.com","subject":"Re: [PATCH] all: new command used for multi-repo operations","fromName":"Lars Hjemli","fromEmail":"hjemli@gmail.com","sentAt":"2013-01-23T08:46:04Z","receivedAt":"2013-01-23T08:46:04Z","isPatch":true,"sender":{"key":"hjemli@gmail.com","avatar":null},"body":"On Wed, Jan 23, 2013 at 9:39 AM, Duy Nguyen wrote:\n> On Wed, Jan 23, 2013 at 4:10 AM, Lars Hjemli wrote:\n>> When working with multiple, unrelated (or loosly related) git repos,\n>> there is often a need to locate all repos with uncommitted work and\n>> perform some action on them (say, commit and push). Before this patch,\n>> such tasks would require manually visiting all repositories, running\n>> `git status` within each one and then decide what to do next.\n>>\n>> This mundane task can now be automated by e.g. `git all --dirty status`,\n>> which will find all git repositories below the current directory (even\n>> nested ones), check if they are dirty (as defined by `git diff --quiet &&\n>> git diff --cached --quiet`), and for each dirty repo print the path to the\n>> repo and then execute `git status` within the repo.\n>\n> I think it should leave out the execute part. The command, say\n> ls-repo, lists repositories in specified state. The execute part could\n> be easily done by\n>\n> xargs -I{} git --git-dir={} status blah\n\nNot so easily on windows, which I need to use at $WORK :(\n\n--\nlarsh\n"},{"id":"207574","messageId":"CAFXTnz5pG8ifPvMp8xin3xSikGT9iLoUfsrqpepxDDTCMZ2wRQ@mail.gmail.com","threadId":"32695","inReplyTo":"CAFXTnz51czE5iS_pgZyU7SdUwrmZcmLxjFGpCGVhLGJFvW=HRQ@mail.gmail.com","subject":"[PATCH] all: new command used for multi-repo operations","fromName":"Lars Hjemli","fromEmail":"hjemli@gmail.com","sentAt":"2013-01-23T08:52:52Z","receivedAt":"2013-01-23T08:52:52Z","isPatch":true,"sender":{"key":"hjemli@gmail.com","avatar":null},"body":"[*git@vger.kernel.org accidentally dropped from cc *]\n\nOn Wed, Jan 23, 2013 at 7:52 AM, Junio C Hamano wrote:\n> Lars Hjemli writes:\n>\n>> +static int walk(struct strbuf *path, int argc, const char **argv)\n>> +{\n>> + DIR *dir;\n>> + struct dirent *ent;\n>> + size_t len;\n>> +\n>> + dir = opendir(path->buf);\n>> + if (!dir)\n>> + return errno;\n>> + strbuf_addstr(path, \"/\");\n>> + len = path->len;\n>> + while ((ent = readdir(dir))) {\n>> + if (!strcmp(ent->d_name, \".\") || !strcmp(ent->d_name, \"..\"))\n>> + continue;\n>> + if (!strcmp(ent->d_name, \".git\")) {\n>\n> This only looks for the top of working tree. Have you considered if\n> this \"iterate over directories and list git repositories in them\"\n> may be useful for collection of bare repositories, and if it is, how\n> to go about implementing the discovery process?\n\nYes, occasionally I've needed this, but implementing it in my original\nshell script was cumbersome. Doing it in C should be as simple as\ninvoking is_git_directory() on each subdir.\n\n>\n>> + if (ent->d_type != DT_DIR)\n>> + continue;\n>\n> I think this is wrong.\n>\n> On platforms that need a NO_D_TYPE_IN_DIRENT build, your compilation\n> may fail here (you would need to lstat() it yourself). See how\n> dir.c does this without ugly #ifdef's in the code, especially around\n> the use of get_dtype() and DTYPE() macro.\n>\n\nThanks for the pointer, will fix.\n\n--\nlarsh\n"}]}