threads / rfc / 48155

RFC patch, 5 partsKeep all info in command-list.txt in git binary

Subject: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

## tl;dr

137 messages between Mar 26, 2018 and May 13, 2018. Diffs are folded; open one to read it.

replies: 136people: 8as markdown or json

Nguyễn Thái Ngọc Duy· Mar 26, 2018, 16:55 UTC · lore

This is pretty rough but I'd like to see how people feel about this first.

I notice we have two places for command classification. One in command-list.txt, one in __git_list_porcelain_commands() in git-completion.bash. People who are following nd/parseopt-completion probably know that I'm try to reduce duplication in this script as much as possible, this is another step towards that.

By keeping all information of command-list.txt in git binary, we could provide the porcelain list to git-completion.bash via "git --list-cmds=porcelain", so we don't neeed a separate command classification in git-completion.bash anymore.

Because we have all command synopsis as a side effect, we could now support "git help -a --verbose" which prints something like "git help", a command name and a description, but we could do it for _all_ recognized commands. This could help people look for a command even if we don't provide "git appropos".

PS. Elsewhere I introduced --list-builtin-cmds, which should become --list-cmds=builtin if this series seems like a good idea to move forward.

Nguyễn Thái Ngọc Duy (5):
  git.c: convert --list-builtins to --list-cmds=builtins
  git.c: implement --list-cmds=all and use it in git-completion.bash
  generate-cmdlist.sh: keep all information in common-cmds.h
  git.c: implement --list-cmds=porcelain
  help: add "-a --verbose" to list all commands with synopsis
 Documentation/git-help.txt             |   4 +-
 builtin/help.c                         |   6 ++
 contrib/completion/git-completion.bash |  94 +-----------------
 generate-cmdlist.sh                    |  46 ++++++---
 git.c                                  |  11 ++-
 help.c                                 | 131 +++++++++++++++++++++++--
 help.h                                 |   3 +
 t/t0012-help.sh                        |   2 +-
 t/t9902-completion.sh                  |   4 +-
 9 files changed, 187 insertions(+), 114 deletions(-)
-- 
2.17.0.rc0.348.gd5a49e0b6f
Nguyễn Thái Ngọc Duy· Mar 26, 2018, 16:55 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH/RFC 2/5] git.c: implement --list-cmds=all and use it in git-completion.bash

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  |  2 ++
 help.c                                 | 15 +++++++++++++++
 help.h                                 |  1 +
 4 files changed, 19 insertions(+), 1 deletion(-)
Show changes to 4 files +19 −1

contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index c7957f0a90..e0f545819d 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -833,7 +833,7 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git help -a|egrep '^  [a-zA-Z0-9]'
+		git --list-cmds=all
 	fi
 }
 
diff --git a/git.c b/git.c
index f350002260..2e0c5e17e2 100644
--- a/git.c
+++ b/git.c
@@ -208,6 +208,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
 			if (!strcmp(cmd, "builtins"))
 				list_builtins();
+			else if (!strcmp(cmd, "all"))
+				list_all_cmds();
 			else
 				die("unsupported command listing type '%s'", cmd);
 			exit(0);
diff --git a/help.c b/help.c
index 60071a9bea..f3f35dfbb1 100644
--- a/help.c
+++ b/help.c
@@ -228,6 +228,21 @@ void list_common_cmds_help(void)
 	}
 }
 
+void list_all_cmds(void)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < main_cmds.cnt; i++)
+		puts(main_cmds.names[i]->name);
+	for (i = 0; i < other_cmds.cnt; i++)
+		puts(other_cmds.names[i]->name);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index b21d7c94e8..0bf29f8dc5 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.rc0.348.gd5a49e0b6f
Eric Sunshine· Apr 9, 2018, 03:32 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH/RFC 2/5] git.c: implement --list-cmds=all and use it in git-completion.bash

On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 17 quoted lines
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> diff --git a/help.c b/help.c
> @@ -228,6 +228,21 @@ void list_common_cmds_help(void)
> +void list_all_cmds(void)
> +{
> +       struct cmdnames main_cmds, other_cmds;
> +       int i;
> +
> +       memset(&main_cmds, 0, sizeof(main_cmds));
> +       memset(&other_cmds, 0, sizeof(other_cmds));
> +       load_command_list("git-", &main_cmds, &other_cmds);
> +
> +       for (i = 0; i < main_cmds.cnt; i++)
> +               puts(main_cmds.names[i]->name);
> +       for (i = 0; i < other_cmds.cnt; i++)
> +               puts(other_cmds.names[i]->name);

clean_cmdnames(&main_cmds); clean_cmdnames(&other_cmds);

> +}
Nguyễn Thái Ngọc Duy· Mar 26, 2018, 16:55 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH/RFC 1/5] git.c: convert --list-builtins to --list-cmds=builtins

Even if this is a hidden option, let's make it a bit more generic since we're introducing more listing types.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 git.c           | 7 +++++--
 t/t0012-help.sh | 2 +-
 2 files changed, 6 insertions(+), 3 deletions(-)
Show changes to 2 files +6 −3

git.c, t/t0012-help.sh

diff --git a/git.c b/git.c
index ceaa58ef40..f350002260 100644
--- a/git.c
+++ b/git.c
@@ -205,8 +205,11 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			}
 			(*argv)++;
 			(*argc)--;
-		} else if (!strcmp(cmd, "--list-builtins")) {
-			list_builtins();
+		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
+			if (!strcmp(cmd, "builtins"))
+				list_builtins();
+			else
+				die("unsupported command listing type '%s'", cmd);
 			exit(0);
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 487b92a5de..fd2a7f27dc 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -50,7 +50,7 @@ test_expect_success "--help does not work for guides" "
 "
 
 test_expect_success 'generate builtin list' '
-	git --list-builtins >builtins
+	git --list-cmds=builtins >builtins
 '
 
 while read builtin
-- 
2.17.0.rc0.348.gd5a49e0b6f
Nguyễn Thái Ngọc Duy· Mar 26, 2018, 16:55 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH/RFC 3/5] generate-cmdlist.sh: keep all information in common-cmds.h

common-cmds.h is used to extract the list of common commands (by group) and a one-line summary of each command. Some information is dropped, for example command category or summary of other commands. Update generate-cmdlist.sh to keep all the information. The extra info will be used shortly.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 generate-cmdlist.sh | 46 ++++++++++++++++++++++++++++++++++-----------
 help.c              | 43 ++++++++++++++++++++++++++++++++++++------
 2 files changed, 72 insertions(+), 17 deletions(-)
Show changes to 2 files +72 −17

generate-cmdlist.sh, help.c

diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index eeea4b67ea..72235e7296 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -2,9 +2,10 @@
 
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
-	char name[16];
+	char name[32];
 	char help[80];
-	unsigned char group;
+	unsigned int category;
+	unsigned int group;
 };
 
 static const char *common_cmd_groups[] = {"
@@ -23,27 +24,50 @@ sed -n '
 	' "$1"
 printf '};\n\n'
 
+echo "#define GROUP_NONE 0xff /* no common group */"
+echo "#define GROUP_ 0xff /* no common group */"
 n=0
-substnum=
 while read grp
 do
-	echo "^git-..*[ 	]$grp"
-	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
+	echo "#define GROUP_$grp $n"
 	n=$(($n+1))
-done <"$grps" >"$match"
+done <"$grps"
+echo
 
-printf 'static struct cmdname_help common_cmds[] = {\n'
-grep -f "$match" "$1" |
+echo '/*'
+printf 'static const char *cmd_categories[] = {\n'
+grep '^git-' "$1" |
+awk '{print $2;}' |
+sort |
+uniq |
+while read category; do
+	printf '\t\"'$category'\",\n'
+done
+printf '\tNULL\n};\n\n'
+echo '*/'
+
+n=0
+grep '^git-' "$1" |
+awk '{print $2;}' |
+sort |
+uniq |
+while read category; do
+	echo "#define CAT_$category $n"
+	n=$(($n+1))
+done
+echo
+
+printf 'static struct cmdname_help command_list[] = {\n'
+grep "^git-" "$1" |
 sed 's/^git-//' |
 sort |
-while read cmd tags
+while read cmd category tags
 do
-	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
 	sed -n '
 		/^NAME/,/git-'"$cmd"'/H
 		${
 			x
-			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
+			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), CAT_'$category', GROUP_'$tags' },/
 			p
 		}' "Documentation/git-$cmd.txt"
 done
diff --git a/help.c b/help.c
index f3f35dfbb1..4d07ea3913 100644
--- a/help.c
+++ b/help.c
@@ -190,6 +190,28 @@ void list_commands(unsigned int colopts,
 	}
 }
 
+static void extract_common_cmds(struct cmdname_help **p_common_cmds,
+				int *p_nr)
+{
+	int i, nr = 0;
+	struct cmdname_help *common_cmds;
+
+	ALLOC_ARRAY(common_cmds, ARRAY_SIZE(command_list));
+
+	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
+		const struct cmdname_help *cmd = command_list + i;
+
+		if (cmd->category != CAT_mainporcelain ||
+		    cmd->group == GROUP_NONE)
+			continue;
+
+		common_cmds[nr++] = *cmd;
+	}
+
+	*p_common_cmds = common_cmds;
+	*p_nr = nr;
+}
+
 static int cmd_group_cmp(const void *elem1, const void *elem2)
 {
 	const struct cmdname_help *e1 = elem1;
@@ -206,17 +228,21 @@ void list_common_cmds_help(void)
 {
 	int i, longest = 0;
 	int current_grp = -1;
+	int nr = 0;
+	struct cmdname_help *common_cmds;
+
+	extract_common_cmds(&common_cmds, &nr);
 
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
+	for (i = 0; i < nr; i++) {
 		if (longest < strlen(common_cmds[i].name))
 			longest = strlen(common_cmds[i].name);
 	}
 
-	QSORT(common_cmds, ARRAY_SIZE(common_cmds), cmd_group_cmp);
+	QSORT(common_cmds, nr, cmd_group_cmp);
 
 	puts(_("These are common Git commands used in various situations:"));
 
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
+	for (i = 0; i < nr; i++) {
 		if (common_cmds[i].group != current_grp) {
 			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
 			current_grp = common_cmds[i].group;
@@ -226,6 +252,7 @@ void list_common_cmds_help(void)
 		mput_char(' ', longest - strlen(common_cmds[i].name));
 		puts(_(common_cmds[i].help));
 	}
+	free(common_cmds);
 }
 
 void list_all_cmds(void)
@@ -298,8 +325,9 @@ static const char bad_interpreter_advice[] =
 
 const char *help_unknown_cmd(const char *cmd)
 {
-	int i, n, best_similarity = 0;
+	int i, n, best_similarity = 0, nr_common;
 	struct cmdnames main_cmds, other_cmds;
+	struct cmdname_help *common_cmds;
 
 	memset(&main_cmds, 0, sizeof(main_cmds));
 	memset(&other_cmds, 0, sizeof(other_cmds));
@@ -314,6 +342,8 @@ const char *help_unknown_cmd(const char *cmd)
 	QSORT(main_cmds.names, main_cmds.cnt, cmdname_compare);
 	uniq(&main_cmds);
 
+	extract_common_cmds(&common_cmds, &nr_common);
+
 	/* This abuses cmdname->len for levenshtein distance */
 	for (i = 0, n = 0; i < main_cmds.cnt; i++) {
 		int cmp = 0; /* avoid compiler stupidity */
@@ -328,10 +358,10 @@ const char *help_unknown_cmd(const char *cmd)
 			die(_(bad_interpreter_advice), cmd, cmd);
 
 		/* Does the candidate appear in common_cmds list? */
-		while (n < ARRAY_SIZE(common_cmds) &&
+		while (n < nr_common &&
 		       (cmp = strcmp(common_cmds[n].name, candidate)) < 0)
 			n++;
-		if ((n < ARRAY_SIZE(common_cmds)) && !cmp) {
+		if ((n < nr_common) && !cmp) {
 			/* Yes, this is one of the common commands */
 			n++; /* use the entry from common_cmds[] */
 			if (starts_with(candidate, cmd)) {
@@ -344,6 +374,7 @@ const char *help_unknown_cmd(const char *cmd)
 		main_cmds.names[i]->len =
 			levenshtein(cmd, candidate, 0, 2, 1, 3) + 1;
 	}
+	FREE_AND_NULL(common_cmds);
 
 	QSORT(main_cmds.names, main_cmds.cnt, levenshtein_compare);
 
-- 
2.17.0.rc0.348.gd5a49e0b6f
Eric Sunshine· Apr 9, 2018, 04:59 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH/RFC 3/5] generate-cmdlist.sh: keep all information in common-cmds.h

On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 21 quoted lines
> common-cmds.h is used to extract the list of common commands (by
> group) and a one-line summary of each command. Some information is
> dropped, for example command category or summary of other commands.
> Update generate-cmdlist.sh to keep all the information. The extra info
> will be used shortly.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> @@ -2,9 +2,10 @@
>  struct cmdname_help {
> -       char name[16];
> +       char name[32];
>         char help[80];
> -       unsigned char group;
> +       unsigned int category;
> +       unsigned int group;
>  };
> @@ -23,27 +24,50 @@ sed -n '
> +echo "#define GROUP_NONE 0xff /* no common group */"
> +echo "#define GROUP_ 0xff /* no common group */"
Meh, this "GROUP_" alias of "GROUP_NONE" isn't so nice.
Show 10 quoted lines
>  n=0
> -substnum=
>  while read grp
>  do
> -       echo "^git-..*[         ]$grp"
> -       substnum="$substnum${substnum:+;}s/[    ]$grp/$n/"
> +       echo "#define GROUP_$grp $n"
>         n=$(($n+1))
> -done <"$grps" >"$match"
> +done <"$grps"

This patch drops all use of the file $match. Earlier in this script, not seen in the context, are a couple references to $match which ought to be adjusted to take its retirement into account:

    match=match$$.tmp
    trap "rm -f '$grps' '$match'" 0 1 2 3 15

However, I'm concerned that this change may be going in the wrong direction. A line in "### command list" section looks like this:

    command-name  category [deprecated] [common]

Although we don't currently have any commands marked with tag "deprecated", we very well may have some day. More generally, new optional or required tags may be added in the future. As such, the line format is relatively free-form. Current clients don't even care in what order the tags appears (following 'category') nor how many tags there are. The new code added by this patch, however, is far less flexible and accommodating since it assumes hard-coded columns for the tags (and doesn't even take 'deprecated' into account).

The point of the $match file was to be able to extract only lines which mentioned one of the "common groups", and the point of the 'substnum' transformation was to transform the group name into a group number -- both of these operations were done without caring about the exact column the "common group" tag occupied.

Obviously, one option for addressing this concern would be to change the definition to make the tag columns fixed and non-optional, which would allow the simpler implementation used by this patch. Doing so may require fixing other consumers of command-list.txt (though, I'm pretty sure existing consumers wouldn't be bothered).

(Perl would be an obvious good choice for retaining the current relatively free-form line definition without having to jump through hoops in the shell. Unfortunately, though, a Perl dependency in the build system can be problematic[1].)

[1]: https://public-inbox.org/git/1440365469-9928-1-git-send-email-sunshine@sunshineco.com/
Show 5 quoted lines
> -printf 'static struct cmdname_help common_cmds[] = {\n'
> -grep -f "$match" "$1" |
> +echo '/*'
> +printf 'static const char *cmd_categories[] = {\n'
> +grep '^git-' "$1" |

This "grep '^git-'" (and those below) misses some commands, such as "gitk" and "gitweb". Is that intentional? If not, then you'll probably need to grab lines following "### command list", as is done earlier in the script. Same comment for the other couple grep's later in the patch.

> +awk '{print $2;}' |

At one time, Junio expressed concerns[2] about having an 'awk' dependency in the build system (in fact, with regards to this same generation process). Whether he still has such concerns is unknown, but it should be easy enough to avoid it here (and below).

[2]: https://public-inbox.org/git/20150519004356.GA12854@flurp.local/
Show 23 quoted lines
> +sort |
> +uniq |
> +while read category; do
> +       printf '\t\"'$category'\",\n'
> +done
> +printf '\tNULL\n};\n\n'
> +echo '*/'
> diff --git a/help.c b/help.c
> @@ -190,6 +190,28 @@ void list_commands(unsigned int colopts,
> +static void extract_common_cmds(struct cmdname_help **p_common_cmds,
> +                               int *p_nr)
> +{
> +       int i, nr = 0;
> +       struct cmdname_help *common_cmds;
> +
> +       ALLOC_ARRAY(common_cmds, ARRAY_SIZE(command_list));
> +
> +       for (i = 0; i < ARRAY_SIZE(command_list); i++) {
> +               const struct cmdname_help *cmd = command_list + i;
> +
> +               if (cmd->category != CAT_mainporcelain ||
> +                   cmd->group == GROUP_NONE)
> +                       continue;

Is the CAT_mainporcelain condition necessary? Before this patch, the command list would contain only commands with an associated group, so it seems that you could get by just with the GROUP_NONE condition.

Show 7 quoted lines
> +
> +               common_cmds[nr++] = *cmd;
> +       }
> +
> +       *p_common_cmds = common_cmds;
> +       *p_nr = nr;
> +}
Eric Sunshine· Apr 9, 2018, 05:12 UTC · re: Eric Sunshine · lore

Re: [PATCH/RFC 3/5] generate-cmdlist.sh: keep all information in common-cmds.h

On Mon, Apr 9, 2018 at 12:59 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 25 quoted lines
> However, I'm concerned that this change may be going in the wrong
> direction. A line in "### command list" section looks like this:
>
>     command-name  category [deprecated] [common]
>
> Although we don't currently have any commands marked with tag
> "deprecated", we very well may have some day. More generally, new
> optional or required tags may be added in the future. As such, the
> line format is relatively free-form. Current clients don't even care
> in what order the tags appears (following 'category') nor how many
> tags there are. The new code added by this patch, however, is far less
> flexible and accommodating since it assumes hard-coded columns for the
> tags (and doesn't even take 'deprecated' into account).
>
> The point of the $match file was to be able to extract only lines
> which mentioned one of the "common groups", and the point of the
> 'substnum' transformation was to transform the group name into a group
> number -- both of these operations were done without caring about the
> exact column the "common group" tag occupied.
>
> Obviously, one option for addressing this concern would be to change
> the definition to make the tag columns fixed and non-optional, which
> would allow the simpler implementation used by this patch. Doing so
> may require fixing other consumers of command-list.txt (though, I'm
> pretty sure existing consumers wouldn't be bothered).

I should follow up by saying that, although the current relatively free-form line definition is nice due to its flexibility, considering how infrequently command-list.txt is edited and how much more complex the script is to support that flexibility, changing to a definition in which tags are required and at fixed columns seems like the pragmatic thing to do.

Duy Nguyen· Apr 9, 2018, 16:16 UTC · re: Eric Sunshine · lore

Re: [PATCH/RFC 3/5] generate-cmdlist.sh: keep all information in common-cmds.h

On Mon, Apr 9, 2018 at 6:59 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 25 quoted lines
> On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy
> <pclouds@gmail.com> wrote:
>> common-cmds.h is used to extract the list of common commands (by
>> group) and a one-line summary of each command. Some information is
>> dropped, for example command category or summary of other commands.
>> Update generate-cmdlist.sh to keep all the information. The extra info
>> will be used shortly.
>>
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> ---
>> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
>> @@ -2,9 +2,10 @@
>>  struct cmdname_help {
>> -       char name[16];
>> +       char name[32];
>>         char help[80];
>> -       unsigned char group;
>> +       unsigned int category;
>> +       unsigned int group;
>>  };
>> @@ -23,27 +24,50 @@ sed -n '
>> +echo "#define GROUP_NONE 0xff /* no common group */"
>> +echo "#define GROUP_ 0xff /* no common group */"
>
> Meh, this "GROUP_" alias of "GROUP_NONE" isn't so nice.

Yeah. I don't want to mess too much with shell script. I wonder if we should instead kill this script and extend Documentation/cmd-list.perl to handle this task too. It would be much nicer to write and maintain the script. The downside is NO_PERL builds will have no commands in "git help".

-- 
Duy
Duy Nguyen· Apr 15, 2018, 16:04 UTC · re: Eric Sunshine · lore

Re: [PATCH/RFC 3/5] generate-cmdlist.sh: keep all information in common-cmds.h

On Mon, Apr 9, 2018 at 6:59 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 8 quoted lines
>> +awk '{print $2;}' |
>
> At one time, Junio expressed concerns[2] about having an 'awk'
> dependency in the build system (in fact, with regards to this same
> generation process). Whether he still has such concerns is unknown,
> but it should be easy enough to avoid it here (and below).
>
> [2]: https://public-inbox.org/git/20150519004356.GA12854@flurp.local/

I'll stick with awk to avoid too much headache with regular expressions (replacements are welcome though). We do use awk in our test suite so it should be ok (who builds git and runs it without testing?)

-- 
Duy
Nguyễn Thái Ngọc Duy· Mar 26, 2018, 16:55 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH/RFC 5/5] help: add "-a --verbose" to list all commands with synopsis

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-help.txt |  4 ++-
 builtin/help.c             |  6 ++++
 help.c                     | 61 ++++++++++++++++++++++++++++++++++++++
 help.h                     |  1 +
 4 files changed, 71 insertions(+), 1 deletion(-)
Show changes to 4 files +71 −1

Documentation/git-help.txt, builtin/help.c, help.c, help.h

diff --git a/Documentation/git-help.txt b/Documentation/git-help.txt
index 40d328a4b3..a40fc38d8b 100644
--- a/Documentation/git-help.txt
+++ b/Documentation/git-help.txt
@@ -8,7 +8,7 @@ git-help - Display help information about Git
 SYNOPSIS
 --------
 [verse]
-'git help' [-a|--all] [-g|--guide]
+'git help' [-a|--all [--verbose]] [-g|--guide]
 	   [-i|--info|-m|--man|-w|--web] [COMMAND|GUIDE]
 
 DESCRIPTION
@@ -42,6 +42,8 @@ OPTIONS
 --all::
 	Prints all the available commands on the standard output. This
 	option overrides any given command or guide name.
+	When used with `--verbose` print description for all recognized
+	commands.
 
 -g::
 --guides::
diff --git a/builtin/help.c b/builtin/help.c
index 598867cfea..a371199674 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -36,6 +36,7 @@ static const char *html_path;
 
 static int show_all = 0;
 static int show_guides = 0;
+static int verbose;
 static unsigned int colopts;
 static enum help_format help_format = HELP_FORMAT_NONE;
 static int exclude_guides;
@@ -48,6 +49,7 @@ static struct option builtin_help_options[] = {
 			HELP_FORMAT_WEB),
 	OPT_SET_INT('i', "info", &help_format, N_("show info page"),
 			HELP_FORMAT_INFO),
+	OPT__VERBOSE(&verbose, N_("print command description")),
 	OPT_END(),
 };
 
@@ -463,6 +465,10 @@ int cmd_help(int argc, const char **argv, const char *prefix)
 
 	if (show_all) {
 		git_config(git_help_config, NULL);
+		if (verbose) {
+			list_all_cmds_help();
+			return 0;
+		}
 		printf(_("usage: %s%s"), _(git_usage_string), "\n\n");
 		load_command_list("git-", &main_cmds, &other_cmds);
 		list_commands(colopts, &main_cmds, &other_cmds);
diff --git a/help.c b/help.c
index cacd8249bb..afbf98c241 100644
--- a/help.c
+++ b/help.c
@@ -282,6 +282,67 @@ void list_porcelain_cmds(void)
 	}
 }
 
+static int cmd_category_cmp(const void *elem1, const void *elem2)
+{
+	const struct cmdname_help *e1 = elem1;
+	const struct cmdname_help *e2 = elem2;
+
+	if (e1->category < e2->category)
+		return -1;
+	if (e1->category > e2->category)
+		return 1;
+	return strcmp(e1->name, e2->name);
+}
+
+static const char *get_category_name(unsigned int category)
+{
+	switch (category) {
+	case CAT_ancillaryinterrogators: return _("Ancillary interrogators");
+	case CAT_ancillarymanipulators: return _("Ancillary manipulators");
+	case CAT_foreignscminterface: return _("Foreign SCM interface");
+	case CAT_mainporcelain: return _("Main porcelain");
+	case CAT_plumbinginterrogators: return _("Plumbing interrogators");
+	case CAT_plumbingmanipulators: return _("Plumbing interrogators");
+	case CAT_purehelpers: return _("Pure helpers");
+	case CAT_synchelpers: return _("Sync helpers");
+	case CAT_synchingrepositories: return _("Synching repositories");
+	default:
+		die("BUG: unknown command category %u", category);
+	}
+}
+
+void list_all_cmds_help(void)
+{
+	int i, longest = 0;
+	int current_category = -1;
+	int nr = ARRAY_SIZE(command_list);
+	struct cmdname_help *cmds = command_list;
+
+	for (i = 0; i < nr; i++) {
+		struct cmdname_help *cmd = cmds + i;
+
+		if (longest < strlen(cmd->name))
+			longest = strlen(cmd->name);
+	}
+
+	QSORT(cmds, nr, cmd_category_cmp);
+
+	puts(_("These are all Git commands:"));
+
+	for (i = 0; i < nr; i++) {
+		struct cmdname_help *cmd = cmds + i;
+
+		if (cmd->category != current_category) {
+			current_category = cmd->category;
+			printf("\n%s:\n", get_category_name(current_category));
+		}
+
+		printf("   %s   ", cmd->name);
+		mput_char(' ', longest - strlen(cmd->name));
+		puts(_(cmd->help));
+	}
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 33e2210ebd..62449f1b7e 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds_help(void);
 extern void list_all_cmds(void);
 extern void list_porcelain_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
-- 
2.17.0.rc0.348.gd5a49e0b6f
Eric Sunshine· Apr 9, 2018, 05:08 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH/RFC 5/5] help: add "-a --verbose" to list all commands with synopsis

On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 13 quoted lines
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> diff --git a/help.c b/help.c
> @@ -282,6 +282,67 @@ void list_porcelain_cmds(void)
> +static const char *get_category_name(unsigned int category)
> +{
> +       switch (category) {
> +       case CAT_ancillaryinterrogators: return _("Ancillary interrogators");
> +       case CAT_ancillarymanipulators: return _("Ancillary manipulators");
> +       case CAT_foreignscminterface: return _("Foreign SCM interface");
> +       case CAT_mainporcelain: return _("Main porcelain");
> +       case CAT_plumbinginterrogators: return _("Plumbing interrogators");
> +       case CAT_plumbingmanipulators: return _("Plumbing interrogators");
s/interrogators"/manipulators"/
Show 6 quoted lines
> +       case CAT_purehelpers: return _("Pure helpers");
> +       case CAT_synchelpers: return _("Sync helpers");
> +       case CAT_synchingrepositories: return _("Synching repositories");
> +       default:
> +               die("BUG: unknown command category %u", category);
> +       }
Junio C Hamano· Apr 9, 2018, 09:47 UTC · re: Eric Sunshine · lore

Re: [PATCH/RFC 5/5] help: add "-a --verbose" to list all commands with synopsis

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 24 quoted lines
> On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy
> <pclouds@gmail.com> wrote:
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> ---
>> diff --git a/help.c b/help.c
>> @@ -282,6 +282,67 @@ void list_porcelain_cmds(void)
>> +static const char *get_category_name(unsigned int category)
>> +{
>> +       switch (category) {
>> +       case CAT_ancillaryinterrogators: return _("Ancillary interrogators");
>> +       case CAT_ancillarymanipulators: return _("Ancillary manipulators");
>> +       case CAT_foreignscminterface: return _("Foreign SCM interface");
>> +       case CAT_mainporcelain: return _("Main porcelain");
>> +       case CAT_plumbinginterrogators: return _("Plumbing interrogators");
>> +       case CAT_plumbingmanipulators: return _("Plumbing interrogators");
>
> s/interrogators"/manipulators"/
>
>> +       case CAT_purehelpers: return _("Pure helpers");
>> +       case CAT_synchelpers: return _("Sync helpers");
>> +       case CAT_synchingrepositories: return _("Synching repositories");
>> +       default:
>> +               die("BUG: unknown command category %u", category);
>> +       }

Somehow this screams "an array of strings" at me. Aren't this CAT_things small and dense enum?

Eric Sunshine· Apr 9, 2018, 09:55 UTC · re: Junio C Hamano · lore

Re: [PATCH/RFC 5/5] help: add "-a --verbose" to list all commands with synopsis

On Mon, Apr 9, 2018 at 5:47 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 19 quoted lines
> Eric Sunshine <sunshine@sunshineco.com> writes:
>> On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy
>> <pclouds@gmail.com> wrote:
>>> +       switch (category) {
>>> +       case CAT_ancillaryinterrogators: return _("Ancillary interrogators");
>>> +       case CAT_ancillarymanipulators: return _("Ancillary manipulators");
>>> +       case CAT_foreignscminterface: return _("Foreign SCM interface");
>>> +       case CAT_mainporcelain: return _("Main porcelain");
>>> +       case CAT_plumbinginterrogators: return _("Plumbing interrogators");
>>> +       case CAT_plumbingmanipulators: return _("Plumbing interrogators");
>>
>> s/interrogators"/manipulators"/
>>
>>> +       case CAT_purehelpers: return _("Pure helpers");
>>> +       case CAT_synchelpers: return _("Sync helpers");
>>> +       case CAT_synchingrepositories: return _("Synching repositories");
>
> Somehow this screams "an array of strings" at me.  Aren't this
> CAT_things small and dense enum?

Duy's modified generate-cmdlist.sh does actually output an array of strings for these, but the (generated) array is commented out in this RFC. I suppose the reason it's not presently used is because the array looks like this:

static const char *cmd_categories[] = {
    "ancillaryinterrogators",
    "ancillarymanipulators",
    "foreignscminterface",
    "mainporcelain",
    "plumbinginterrogators",
    "plumbingmanipulators",
    "purehelpers",
    "synchelpers",
    "synchingrepositories",
     NULL
};

which doesn't give quite the human-friendly output he'd like. The series is RFC, after all.

A possible approach to fix it would be to add a new "### categories" section to command-list.txt which associates those category tags ("ancillaryinterrogators") with human-readable counterparts ("Ancillary interrogators").

Duy Nguyen· Apr 9, 2018, 15:15 UTC · re: Eric Sunshine · lore

Re: [PATCH/RFC 5/5] help: add "-a --verbose" to list all commands with synopsis

On Mon, Apr 9, 2018 at 11:55 AM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 41 quoted lines
> On Mon, Apr 9, 2018 at 5:47 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Eric Sunshine <sunshine@sunshineco.com> writes:
>>> On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy
>>> <pclouds@gmail.com> wrote:
>>>> +       switch (category) {
>>>> +       case CAT_ancillaryinterrogators: return _("Ancillary interrogators");
>>>> +       case CAT_ancillarymanipulators: return _("Ancillary manipulators");
>>>> +       case CAT_foreignscminterface: return _("Foreign SCM interface");
>>>> +       case CAT_mainporcelain: return _("Main porcelain");
>>>> +       case CAT_plumbinginterrogators: return _("Plumbing interrogators");
>>>> +       case CAT_plumbingmanipulators: return _("Plumbing interrogators");
>>>
>>> s/interrogators"/manipulators"/
>>>
>>>> +       case CAT_purehelpers: return _("Pure helpers");
>>>> +       case CAT_synchelpers: return _("Sync helpers");
>>>> +       case CAT_synchingrepositories: return _("Synching repositories");
>>
>> Somehow this screams "an array of strings" at me.  Aren't this
>> CAT_things small and dense enum?
>
> Duy's modified generate-cmdlist.sh does actually output an array of
> strings for these, but the (generated) array is commented out in this
> RFC. I suppose the reason it's not presently used is because the array
> looks like this:
>
> static const char *cmd_categories[] = {
>     "ancillaryinterrogators",
>     "ancillarymanipulators",
>     "foreignscminterface",
>     "mainporcelain",
>     "plumbinginterrogators",
>     "plumbingmanipulators",
>     "purehelpers",
>     "synchelpers",
>     "synchingrepositories",
>      NULL
> };
>
> which doesn't give quite the human-friendly output he'd like. The
> series is RFC, after all.
Yep.
> A possible approach to fix it would be to add a new "### categories"
> section to command-list.txt which associates those category tags
> ("ancillaryinterrogators") with human-readable counterparts
> ("Ancillary interrogators").

Or extract the headlines from git.txt but that's not easy since it's not consistent. We could manually recreate the same grouping as in git.txt too, it's probably nicer than just printing groups sorted by category id.

-- 
Duy
Nguyễn Thái Ngọc Duy· Mar 26, 2018, 16:55 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH/RFC 4/5] git.c: implement --list-cmds=porcelain

This is useful for git-completion.bash because it needs this set of commands. Right now we have to maintain a separate command category in there.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash | 94 ++------------------------
 git.c                                  |  2 +
 help.c                                 | 12 ++++
 help.h                                 |  1 +
 t/t9902-completion.sh                  |  4 +-
 5 files changed, 20 insertions(+), 93 deletions(-)
Show changes to 5 files +20 −93

contrib/completion/git-completion.bash, git.c, help.c, help.h, t/t9902-completion.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index e0f545819d..d711a9b53a 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -833,14 +833,15 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git --list-cmds=all
+		git --list-cmds=$1
 	fi
 }
 
 __git_list_all_commands ()
 {
 	local i IFS=" "$'\n'
-	for i in $(__git_commands)
+	local category=${1-all}
+	for i in $(__git_commands $category)
 	do
 		case $i in
 		*--*)             : helper pattern;;
@@ -856,98 +857,11 @@ __git_compute_all_commands ()
 	__git_all_commands=$(__git_list_all_commands)
 }
 
-__git_list_porcelain_commands ()
-{
-	local i IFS=" "$'\n'
-	__git_compute_all_commands
-	for i in $__git_all_commands
-	do
-		case $i in
-		*--*)             : helper pattern;;
-		applymbox)        : ask gittus;;
-		applypatch)       : ask gittus;;
-		archimport)       : import;;
-		cat-file)         : plumbing;;
-		check-attr)       : plumbing;;
-		check-ignore)     : plumbing;;
-		check-mailmap)    : plumbing;;
-		check-ref-format) : plumbing;;
-		checkout-index)   : plumbing;;
-		column)           : internal helper;;
-		commit-tree)      : plumbing;;
-		count-objects)    : infrequent;;
-		credential)       : credentials;;
-		credential-*)     : credentials helper;;
-		cvsexportcommit)  : export;;
-		cvsimport)        : import;;
-		cvsserver)        : daemon;;
-		daemon)           : daemon;;
-		diff-files)       : plumbing;;
-		diff-index)       : plumbing;;
-		diff-tree)        : plumbing;;
-		fast-import)      : import;;
-		fast-export)      : export;;
-		fsck-objects)     : plumbing;;
-		fetch-pack)       : plumbing;;
-		fmt-merge-msg)    : plumbing;;
-		for-each-ref)     : plumbing;;
-		hash-object)      : plumbing;;
-		http-*)           : transport;;
-		index-pack)       : plumbing;;
-		init-db)          : deprecated;;
-		local-fetch)      : plumbing;;
-		ls-files)         : plumbing;;
-		ls-remote)        : plumbing;;
-		ls-tree)          : plumbing;;
-		mailinfo)         : plumbing;;
-		mailsplit)        : plumbing;;
-		merge-*)          : plumbing;;
-		mktree)           : plumbing;;
-		mktag)            : plumbing;;
-		pack-objects)     : plumbing;;
-		pack-redundant)   : plumbing;;
-		pack-refs)        : plumbing;;
-		parse-remote)     : plumbing;;
-		patch-id)         : plumbing;;
-		prune)            : plumbing;;
-		prune-packed)     : plumbing;;
-		quiltimport)      : import;;
-		read-tree)        : plumbing;;
-		receive-pack)     : plumbing;;
-		remote-*)         : transport;;
-		rerere)           : plumbing;;
-		rev-list)         : plumbing;;
-		rev-parse)        : plumbing;;
-		runstatus)        : plumbing;;
-		sh-setup)         : internal;;
-		shell)            : daemon;;
-		show-ref)         : plumbing;;
-		send-pack)        : plumbing;;
-		show-index)       : plumbing;;
-		ssh-*)            : transport;;
-		stripspace)       : plumbing;;
-		symbolic-ref)     : plumbing;;
-		unpack-file)      : plumbing;;
-		unpack-objects)   : plumbing;;
-		update-index)     : plumbing;;
-		update-ref)       : plumbing;;
-		update-server-info) : daemon;;
-		upload-archive)   : plumbing;;
-		upload-pack)      : plumbing;;
-		write-tree)       : plumbing;;
-		var)              : infrequent;;
-		verify-pack)      : infrequent;;
-		verify-tag)       : plumbing;;
-		*) echo $i;;
-		esac
-	done
-}
-
 __git_porcelain_commands=
 __git_compute_porcelain_commands ()
 {
 	test -n "$__git_porcelain_commands" ||
-	__git_porcelain_commands=$(__git_list_porcelain_commands)
+	__git_porcelain_commands=$(__git_list_all_commands porcelain)
 }
 
 # Lists all set config variables starting with the given section prefix,
diff --git a/git.c b/git.c
index 2e0c5e17e2..5b09f77792 100644
--- a/git.c
+++ b/git.c
@@ -210,6 +210,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 				list_builtins();
 			else if (!strcmp(cmd, "all"))
 				list_all_cmds();
+			else if (!strcmp(cmd, "porcelain"))
+				list_porcelain_cmds();
 			else
 				die("unsupported command listing type '%s'", cmd);
 			exit(0);
diff --git a/help.c b/help.c
index 4d07ea3913..cacd8249bb 100644
--- a/help.c
+++ b/help.c
@@ -270,6 +270,18 @@ void list_all_cmds(void)
 		puts(other_cmds.names[i]->name);
 }
 
+void list_porcelain_cmds(void)
+{
+	int i, nr = ARRAY_SIZE(command_list);
+	struct cmdname_help *cmds = command_list;
+
+	for (i = 0; i < nr; i++) {
+		if (cmds[i].category != CAT_mainporcelain)
+			continue;
+		puts(cmds[i].name);
+	}
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 0bf29f8dc5..33e2210ebd 100644
--- a/help.h
+++ b/help.h
@@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds(void);
+extern void list_porcelain_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index e6485feb0a..79a2c6d67d 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1208,8 +1208,6 @@ test_expect_success 'basic' '
 	grep -q "^add \$" out &&
 	# script
 	grep -q "^filter-branch \$" out &&
-	# plumbing
-	! grep -q "^ls-files \$" out &&
 
 	run_completion "git f" &&
 	! grep -q -v "^f" out
@@ -1272,7 +1270,7 @@ test_expect_success 'general options' '
 	test_completion "git --no-r" "--no-replace-objects "
 '
 
-test_expect_success 'general options plus command' '
+test_expect_failure 'general options plus command' '
 	test_completion "git --version check" "checkout " &&
 	test_completion "git --paginate check" "checkout " &&
 	test_completion "git --git-dir=foo check" "checkout " &&
-- 
2.17.0.rc0.348.gd5a49e0b6f
Eric Sunshine· Apr 9, 2018, 05:17 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 13 quoted lines
> This is pretty rough but I'd like to see how people feel about this
> first.
>
> I notice we have two places for command classification. One in
> command-list.txt, one in __git_list_porcelain_commands() in
> git-completion.bash. People who are following nd/parseopt-completion
> probably know that I'm try to reduce duplication in this script as
> much as possible, this is another step towards that.
>
> By keeping all information of command-list.txt in git binary, we could
> provide the porcelain list to git-completion.bash via "git
> --list-cmds=porcelain", so we don't neeed a separate command
> classification in git-completion.bash anymore.
I like the direction this series is taking.
Show 5 quoted lines
> Because we have all command synopsis as a side effect, we could
> now support "git help -a --verbose" which prints something like "git
> help", a command name and a description, but we could do it for _all_
> recognized commands. This could help people look for a command even if
> we don't provide "git appropos".

Nice idea, and you practically get this for free (aside from the the obvious new code) since generate-cmdlist.sh already plucks the summary for each command directly from Documentation/git-*.txt.

Philip Oakley· Apr 11, 2018, 22:06 UTC · re: Eric Sunshine · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

From: "Eric Sunshine" <sunshine@sunshineco.com> Monday, April 09, 2018 6:17 
AM
On Mon, Mar 26, 2018 at 12:55 PM, Nguyễn Thái Ngọc Duy
<pclouds@gmail.com> wrote:
Show 13 quoted lines
> This is pretty rough but I'd like to see how people feel about this
> first.
>
> I notice we have two places for command classification. One in
> command-list.txt, one in __git_list_porcelain_commands() in
> git-completion.bash. People who are following nd/parseopt-completion
> probably know that I'm try to reduce duplication in this script as
> much as possible, this is another step towards that.
>
> By keeping all information of command-list.txt in git binary, we could
> provide the porcelain list to git-completion.bash via "git
> --list-cmds=porcelain", so we don't neeed a separate command
> classification in git-completion.bash anymore.
I like the direction this series is taking.
Show 5 quoted lines
> Because we have all command synopsis as a side effect, we could
> now support "git help -a --verbose" which prints something like "git
> help", a command name and a description, but we could do it for _all_
> recognized commands. This could help people look for a command even if
> we don't provide "git appropos".

Nice idea, and you practically get this for free (aside from the the obvious new code) since generate-cmdlist.sh already plucks the summary for each command directly from Documentation/git-*.txt.

I'm only just catching up, but does/can this series also capture the non-command guides that are available in git so that the 'git help -g' can begin to list them all?

It was something I looked at some years ago (when I added the -g option) but at the time the idea of updating the command-list.txt was too invasive.

Just a thought.
Philip
Duy Nguyen· Apr 14, 2018, 15:44 UTC · re: Philip Oakley · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

On Thu, Apr 12, 2018 at 12:06 AM, Philip Oakley <philipoakley@iee.org> wrote:
> I'm only just catching up, but does/can this series also capture the
> non-command guides that are available in git so that the 'git help -g' can
> begin to list them all?

It currently does not. But I don't see why it should not. This should allow git.txt to list all the guides too, for people who skip "git help" and go hard core mode with "man git". Thanks for bringing this up.

-- 
Duy
Philip Oakley· Apr 15, 2018, 21:21 UTC · re: Duy Nguyen · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

From: "Duy Nguyen" <pclouds@gmail.com> : Saturday, April 14, 2018 4:44 PM
Show 14 quoted lines
> On Thu, Apr 12, 2018 at 12:06 AM, Philip Oakley <philipoakley@iee.org>
> wrote:
>> I'm only just catching up, but does/can this series also capture the
>> non-command guides that are available in git so that the 'git help -g'
>> can
>> begin to list them all?
>
> It currently does not. But I don't see why it should not. This should
> allow git.txt to list all the guides too, for people who skip "git
> help" and go hard core mode with "man git". Thanks for bringing this
> up.
> -- 
> Duy
>
Is that something I should add to my todo to add a 'guide' category etc.?

A quick search of public-inbox suggests https://public-inbox.org/git/1361660761-1932-1-git-send-email-philipoakley@iee.org/ as being where I first made the suggestions, but it got trimmed back to not update (be embedded in) the command-list.txt

Philip
Duy Nguyen· Apr 17, 2018, 16:24 UTC · re: Philip Oakley · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

On Sun, Apr 15, 2018 at 11:21 PM, Philip Oakley <philipoakley@iee.org> wrote:
Show 19 quoted lines
> From: "Duy Nguyen" <pclouds@gmail.com> : Saturday, April 14, 2018 4:44 PM
>
>> On Thu, Apr 12, 2018 at 12:06 AM, Philip Oakley <philipoakley@iee.org>
>> wrote:
>>>
>>> I'm only just catching up, but does/can this series also capture the
>>> non-command guides that are available in git so that the 'git help -g'
>>> can
>>> begin to list them all?
>>
>>
>> It currently does not. But I don't see why it should not. This should
>> allow git.txt to list all the guides too, for people who skip "git
>> help" and go hard core mode with "man git". Thanks for bringing this
>> up.
>> --
>> Duy
>>
> Is that something I should add to my todo to add a 'guide' category etc.?
I added it too [1]. Not sure if you want anything more on top though.
[1] https://public-inbox.org/git/20180415164238.9107-7-pclouds@gmail.com/T/#u
-- 
Duy
Duy Nguyen· Apr 17, 2018, 16:48 UTC · re: Duy Nguyen · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

On Tue, Apr 17, 2018 at 06:24:41PM +0200, Duy Nguyen wrote:
Show 22 quoted lines
> On Sun, Apr 15, 2018 at 11:21 PM, Philip Oakley <philipoakley@iee.org> wrote:
> > From: "Duy Nguyen" <pclouds@gmail.com> : Saturday, April 14, 2018 4:44 PM
> >
> >> On Thu, Apr 12, 2018 at 12:06 AM, Philip Oakley <philipoakley@iee.org>
> >> wrote:
> >>>
> >>> I'm only just catching up, but does/can this series also capture the
> >>> non-command guides that are available in git so that the 'git help -g'
> >>> can
> >>> begin to list them all?
> >>
> >>
> >> It currently does not. But I don't see why it should not. This should
> >> allow git.txt to list all the guides too, for people who skip "git
> >> help" and go hard core mode with "man git". Thanks for bringing this
> >> up.
> >> --
> >> Duy
> >>
> > Is that something I should add to my todo to add a 'guide' category etc.?
> 
> I added it too [1]. Not sure if you want anything more on top though.

The "anything more" that at least I had in mind was something like this. Though I'm not sure if it's a good thing to replace a hand crafted section with an automatedly generated one. This patch on top combines the "SEE ALSO" and "FURTHER DOCUMENT" into one with most of documents/guides are extracted from command-list.txt

-- 8< --
Show changes to 4 files +16 −28

Documentation/Makefile, Documentation/cmd-list.perl, Documentation/git.txt, command-list.txt

diff --git a/Documentation/Makefile b/Documentation/Makefile
index 6232143cb9..3e0ecd2e11 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -292,6 +292,7 @@ doc.dep : $(docdep_prereqs) $(wildcard *.txt) build-docdep.perl
 
 cmds_txt = cmds-ancillaryinterrogators.txt \
 	cmds-ancillarymanipulators.txt \
+	cmds-guide.txt \
 	cmds-mainporcelain.txt \
 	cmds-plumbinginterrogators.txt \
 	cmds-plumbingmanipulators.txt \
diff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl
index 5aa73cfe45..e158bd9b96 100755
--- a/Documentation/cmd-list.perl
+++ b/Documentation/cmd-list.perl
@@ -54,6 +54,7 @@ for (sort <>) {
 
 for my $cat (qw(ancillaryinterrogators
 		ancillarymanipulators
+		guide
 		mainporcelain
 		plumbinginterrogators
 		plumbingmanipulators
diff --git a/Documentation/git.txt b/Documentation/git.txt
index 4767860e72..d60d2ae0c7 100644
--- a/Documentation/git.txt
+++ b/Documentation/git.txt
@@ -808,29 +808,6 @@ The index is also capable of storing multiple entries (called "stages")
 for a given pathname.  These stages are used to hold the various
 unmerged version of a file when a merge is in progress.
 
-FURTHER DOCUMENTATION
----------------------
-
-See the references in the "description" section to get started
-using Git.  The following is probably more detail than necessary
-for a first-time user.
-
-The link:user-manual.html#git-concepts[Git concepts chapter of the
-user-manual] and linkgit:gitcore-tutorial[7] both provide
-introductions to the underlying Git architecture.
-
-See linkgit:gitworkflows[7] for an overview of recommended workflows.
-
-See also the link:howto-index.html[howto] documents for some useful
-examples.
-
-The internals are documented in the
-link:technical/api-index.html[Git API documentation].
-
-Users migrating from CVS may also want to
-read linkgit:gitcvs-migration[7].
-
-
 Authors
 -------
 Git was started by Linus Torvalds, and is currently maintained by Junio
@@ -854,11 +831,16 @@ the Git Security mailing list <git-security@googlegroups.com>.
 
 SEE ALSO
 --------
-linkgit:gittutorial[7], linkgit:gittutorial-2[7],
-linkgit:giteveryday[7], linkgit:gitcvs-migration[7],
-linkgit:gitglossary[7], linkgit:gitcore-tutorial[7],
-linkgit:gitcli[7], link:user-manual.html[The Git User's Manual],
-linkgit:gitworkflows[7]
+
+See the references in the "description" section to get started
+using Git.  The following is probably more detail than necessary
+for a first-time user.
+
+include::cmds-guide.txt[]
+
+See also the link:howto-index.html[howto] documents for some useful
+examples. The internals are documented in the
+link:technical/api-index.html[Git API documentation].
 
 GIT
 ---
diff --git a/command-list.txt b/command-list.txt
index 1835f1a928..f26b8acd52 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -150,10 +150,14 @@ git-whatchanged                         ancillaryinterrogators
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
 gitattributes                           guide
+gitcvs-migration                        guide
+gitcli                                  guide
+gitcore-tutorial                        guide
 giteveryday                             guide
 gitglossary                             guide
 gitignore                               guide
 gitmodules                              guide
 gitrevisions                            guide
 gittutorial                             guide
+gittutorial-2                           guide
 gitworkflows                            guide
-- 8< --
Philip Oakley· Apr 17, 2018, 22:47 UTC · re: Duy Nguyen · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

From: "Duy Nguyen" <pclouds@gmail.com> : Tuesday, April 17, 2018 5:48 PM
Show 27 quoted lines
> On Tue, Apr 17, 2018 at 06:24:41PM +0200, Duy Nguyen wrote:
>> On Sun, Apr 15, 2018 at 11:21 PM, Philip Oakley <philipoakley@iee.org> 
>> wrote:
>> > From: "Duy Nguyen" <pclouds@gmail.com> : Saturday, April 14, 2018 4:44 
>> > PM
>> >
>> >> On Thu, Apr 12, 2018 at 12:06 AM, Philip Oakley <philipoakley@iee.org>
>> >> wrote:
>> >>>
>> >>> I'm only just catching up, but does/can this series also capture the
>> >>> non-command guides that are available in git so that the 'git 
>> >>> help -g'
>> >>> can
>> >>> begin to list them all?
>> >>
>> >>
>> >> It currently does not. But I don't see why it should not. This should
>> >> allow git.txt to list all the guides too, for people who skip "git
>> >> help" and go hard core mode with "man git". Thanks for bringing this
>> >> up.
>> >> --
>> >> Duy
>> >>
>> > Is that something I should add to my todo to add a 'guide' category 
>> > etc.?
>>
>> I added it too [1]. Not sure if you want anything more on top though.
What I've seen is looking good - I've not had as much time as I'd like..

I'm not sure of the status of the git/generate-cmdlist.sh though. Should that also be updated, or did I miss that? -- Philip

Show 113 quoted lines
>
> The "anything more" that at least I had in mind was something like
> this. Though I'm not sure if it's a good thing to replace a hand
> crafted section with an automatedly generated one. This patch on top
> combines the "SEE ALSO" and "FURTHER DOCUMENT" into one with most of
> documents/guides are extracted from command-list.txt
>
> -- 8< --
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 6232143cb9..3e0ecd2e11 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -292,6 +292,7 @@ doc.dep : $(docdep_prereqs) $(wildcard *.txt) 
> build-docdep.perl
>
> cmds_txt = cmds-ancillaryinterrogators.txt \
>  cmds-ancillarymanipulators.txt \
> + cmds-guide.txt \
>  cmds-mainporcelain.txt \
>  cmds-plumbinginterrogators.txt \
>  cmds-plumbingmanipulators.txt \
> diff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl
> index 5aa73cfe45..e158bd9b96 100755
> --- a/Documentation/cmd-list.perl
> +++ b/Documentation/cmd-list.perl
> @@ -54,6 +54,7 @@ for (sort <>) {
>
> for my $cat (qw(ancillaryinterrogators
>  ancillarymanipulators
> + guide
>  mainporcelain
>  plumbinginterrogators
>  plumbingmanipulators
> diff --git a/Documentation/git.txt b/Documentation/git.txt
> index 4767860e72..d60d2ae0c7 100644
> --- a/Documentation/git.txt
> +++ b/Documentation/git.txt
> @@ -808,29 +808,6 @@ The index is also capable of storing multiple entries 
> (called "stages")
> for a given pathname.  These stages are used to hold the various
> unmerged version of a file when a merge is in progress.
>
> -FURTHER DOCUMENTATION
> ----------------------
> -
> -See the references in the "description" section to get started
> -using Git.  The following is probably more detail than necessary
> -for a first-time user.
> -
> -The link:user-manual.html#git-concepts[Git concepts chapter of the
> -user-manual] and linkgit:gitcore-tutorial[7] both provide
> -introductions to the underlying Git architecture.
> -
> -See linkgit:gitworkflows[7] for an overview of recommended workflows.
> -
> -See also the link:howto-index.html[howto] documents for some useful
> -examples.
> -
> -The internals are documented in the
> -link:technical/api-index.html[Git API documentation].
> -
> -Users migrating from CVS may also want to
> -read linkgit:gitcvs-migration[7].
> -
> -
> Authors
> -------
> Git was started by Linus Torvalds, and is currently maintained by Junio
> @@ -854,11 +831,16 @@ the Git Security mailing list 
> <git-security@googlegroups.com>.
>
> SEE ALSO
> --------
> -linkgit:gittutorial[7], linkgit:gittutorial-2[7],
> -linkgit:giteveryday[7], linkgit:gitcvs-migration[7],
> -linkgit:gitglossary[7], linkgit:gitcore-tutorial[7],
> -linkgit:gitcli[7], link:user-manual.html[The Git User's Manual],
> -linkgit:gitworkflows[7]
> +
> +See the references in the "description" section to get started
> +using Git.  The following is probably more detail than necessary
> +for a first-time user.
> +
> +include::cmds-guide.txt[]
> +
> +See also the link:howto-index.html[howto] documents for some useful
> +examples. The internals are documented in the
> +link:technical/api-index.html[Git API documentation].
>
> GIT
> ---
> diff --git a/command-list.txt b/command-list.txt
> index 1835f1a928..f26b8acd52 100644
> --- a/command-list.txt
> +++ b/command-list.txt
> @@ -150,10 +150,14 @@ git-whatchanged 
> ancillaryinterrogators
> git-worktree                            mainporcelain
> git-write-tree                          plumbingmanipulators
> gitattributes                           guide
> +gitcvs-migration                        guide
> +gitcli                                  guide
> +gitcore-tutorial                        guide
> giteveryday                             guide
> gitglossary                             guide
> gitignore                               guide
> gitmodules                              guide
> gitrevisions                            guide
> gittutorial                             guide
> +gittutorial-2                           guide
> gitworkflows                            guide
> -- 8< --
> 
Philip Oakley· Apr 18, 2018, 06:38 UTC · re: Philip Oakley · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

From: "Philip Oakley" <philipoakley@iee.org> : Tuesday, April 17, 2018 11:47
PM
Show 36 quoted lines
> From: "Duy Nguyen" <pclouds@gmail.com> : Tuesday, April 17, 2018 5:48 PM
>> On Tue, Apr 17, 2018 at 06:24:41PM +0200, Duy Nguyen wrote:
>>> On Sun, Apr 15, 2018 at 11:21 PM, Philip Oakley <philipoakley@iee.org>
>>> wrote:
>>> > From: "Duy Nguyen" <pclouds@gmail.com> : Saturday, April 14, 2018 4:44
>>> > PM
>>> >
>>> >> On Thu, Apr 12, 2018 at 12:06 AM, Philip Oakley
>>> >> <philipoakley@iee.org>
>>> >> wrote:
>>> >>>
>>> >>> I'm only just catching up, but does/can this series also capture the
>>> >>> non-command guides that are available in git so that the 'git
>>> >>> help -g'
>>> >>> can
>>> >>> begin to list them all?
>>> >>
>>> >>
>>> >> It currently does not. But I don't see why it should not. This should
>>> >> allow git.txt to list all the guides too, for people who skip "git
>>> >> help" and go hard core mode with "man git". Thanks for bringing this
>>> >> up.
>>> >> --
>>> >> Duy
>>> >>
>>> > Is that something I should add to my todo to add a 'guide' category
>>> > etc.?
>>>
>>> I added it too [1]. Not sure if you want anything more on top though.
>
> What I've seen is looking good - I've not had as much time as I'd like..
>
> I'm not sure of the status of the git/generate-cmdlist.sh though. Should
> that also be updated, or did I miss that?
> --
> Philip

I may be miss-remembering the order that the `git help` determines the list of commands and guides. There was at least one place where the list of commands was generated programatically that I may be confused with (I've not had time to delve into the code :-( --

Show 115 quoted lines
>
>>
>> The "anything more" that at least I had in mind was something like
>> this. Though I'm not sure if it's a good thing to replace a hand
>> crafted section with an automatedly generated one. This patch on top
>> combines the "SEE ALSO" and "FURTHER DOCUMENT" into one with most of
>> documents/guides are extracted from command-list.txt
>>
>> -- 8< --
>> diff --git a/Documentation/Makefile b/Documentation/Makefile
>> index 6232143cb9..3e0ecd2e11 100644
>> --- a/Documentation/Makefile
>> +++ b/Documentation/Makefile
>> @@ -292,6 +292,7 @@ doc.dep : $(docdep_prereqs) $(wildcard *.txt)
>> build-docdep.perl
>>
>> cmds_txt = cmds-ancillaryinterrogators.txt \
>>  cmds-ancillarymanipulators.txt \
>> + cmds-guide.txt \
>>  cmds-mainporcelain.txt \
>>  cmds-plumbinginterrogators.txt \
>>  cmds-plumbingmanipulators.txt \
>> diff --git a/Documentation/cmd-list.perl b/Documentation/cmd-list.perl
>> index 5aa73cfe45..e158bd9b96 100755
>> --- a/Documentation/cmd-list.perl
>> +++ b/Documentation/cmd-list.perl
>> @@ -54,6 +54,7 @@ for (sort <>) {
>>
>> for my $cat (qw(ancillaryinterrogators
>>  ancillarymanipulators
>> + guide
>>  mainporcelain
>>  plumbinginterrogators
>>  plumbingmanipulators
>> diff --git a/Documentation/git.txt b/Documentation/git.txt
>> index 4767860e72..d60d2ae0c7 100644
>> --- a/Documentation/git.txt
>> +++ b/Documentation/git.txt
>> @@ -808,29 +808,6 @@ The index is also capable of storing multiple
>> entries (called "stages")
>> for a given pathname.  These stages are used to hold the various
>> unmerged version of a file when a merge is in progress.
>>
>> -FURTHER DOCUMENTATION
>> ----------------------
>> -
>> -See the references in the "description" section to get started
>> -using Git.  The following is probably more detail than necessary
>> -for a first-time user.
>> -
>> -The link:user-manual.html#git-concepts[Git concepts chapter of the
>> -user-manual] and linkgit:gitcore-tutorial[7] both provide
>> -introductions to the underlying Git architecture.
>> -
>> -See linkgit:gitworkflows[7] for an overview of recommended workflows.
>> -
>> -See also the link:howto-index.html[howto] documents for some useful
>> -examples.
>> -
>> -The internals are documented in the
>> -link:technical/api-index.html[Git API documentation].
>> -
>> -Users migrating from CVS may also want to
>> -read linkgit:gitcvs-migration[7].
>> -
>> -
>> Authors
>> -------
>> Git was started by Linus Torvalds, and is currently maintained by Junio
>> @@ -854,11 +831,16 @@ the Git Security mailing list
>> <git-security@googlegroups.com>.
>>
>> SEE ALSO
>> --------
>> -linkgit:gittutorial[7], linkgit:gittutorial-2[7],
>> -linkgit:giteveryday[7], linkgit:gitcvs-migration[7],
>> -linkgit:gitglossary[7], linkgit:gitcore-tutorial[7],
>> -linkgit:gitcli[7], link:user-manual.html[The Git User's Manual],
>> -linkgit:gitworkflows[7]
>> +
>> +See the references in the "description" section to get started
>> +using Git.  The following is probably more detail than necessary
>> +for a first-time user.
>> +
>> +include::cmds-guide.txt[]
>> +
>> +See also the link:howto-index.html[howto] documents for some useful
>> +examples. The internals are documented in the
>> +link:technical/api-index.html[Git API documentation].
>>
>> GIT
>> ---
>> diff --git a/command-list.txt b/command-list.txt
>> index 1835f1a928..f26b8acd52 100644
>> --- a/command-list.txt
>> +++ b/command-list.txt
>> @@ -150,10 +150,14 @@ git-whatchanged ancillaryinterrogators
>> git-worktree                            mainporcelain
>> git-write-tree                          plumbingmanipulators
>> gitattributes                           guide
>> +gitcvs-migration                        guide
>> +gitcli                                  guide
>> +gitcore-tutorial                        guide
>> giteveryday                             guide
>> gitglossary                             guide
>> gitignore                               guide
>> gitmodules                              guide
>> gitrevisions                            guide
>> gittutorial                             guide
>> +gittutorial-2                           guide
>> gitworkflows                            guide
>> -- 8< --
>>
>
>
Duy Nguyen· Apr 18, 2018, 15:33 UTC · re: Philip Oakley · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

On Wed, Apr 18, 2018 at 12:47 AM, Philip Oakley <philipoakley@iee.org> wrote:
Show 10 quoted lines
>>> > Is that something I should add to my todo to add a 'guide' category >
>>> > etc.?
>>>
>>> I added it too [1]. Not sure if you want anything more on top though.
>
>
> What I've seen is looking good - I've not had as much time as I'd like..
>
> I'm not sure of the status of the git/generate-cmdlist.sh though. Should
> that also be updated, or did I miss that?
Yes it's updated by other patches in the same thread.
-- 
Duy
Philip Oakley· Apr 19, 2018, 07:47 UTC · re: Duy Nguyen · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

From: "Duy Nguyen" <pclouds@gmail.com>
Show 15 quoted lines
> On Wed, Apr 18, 2018 at 12:47 AM, Philip Oakley <philipoakley@iee.org>
> wrote:
>>>> > Is that something I should add to my todo to add a 'guide' category >
>>>> > etc.?
>>>>
>>>> I added it too [1]. Not sure if you want anything more on top though.
>>
>>
>> What I've seen is looking good - I've not had as much time as I'd like..
>>
>> I'm not sure of the status of the git/generate-cmdlist.sh though. Should
>> that also be updated, or did I miss that?
>
> Yes it's updated by other patches in the same thread.
> -- 

Thanks. Hopefully I'll have some time this weekend/coming week as my wife is away.... Philip

Nguyễn Thái Ngọc Duy· Apr 15, 2018, 16:42 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v2 0/6] Keep all info in command-list.txt in git binary

v2 changes
- bug fixes spotted by Eric
- keep 'git help -av' layout close to what's in git.txt
- enable pager for 'git help -av' because it's usually long
- move guide list (aka 'help -g') to command-list.txt
Nguyễn Thái Ngọc Duy (6):
  git.c: convert --list-builtins to --list-cmds=builtins
  git.c: implement --list-cmds=all and use it in git-completion.bash
  generate-cmdlist.sh: keep all information in common-cmds.h
  git.c: implement --list-cmds=porcelain
  help: add "-a --verbose" to list all commands with synopsis
  help: use command-list.txt for the source of guides
 Documentation/git-help.txt             |   4 +-
 Documentation/gitattributes.txt        |   2 +-
 Documentation/gitmodules.txt           |   2 +-
 Documentation/gitrevisions.txt         |   2 +-
 builtin/help.c                         |  39 ++----
 command-list.txt                       |   8 ++
 contrib/completion/git-completion.bash |  96 +--------------
 generate-cmdlist.sh                    |  65 +++++++---
 git.c                                  |  16 ++-
 help.c                                 | 163 ++++++++++++++++++++++++-
 help.h                                 |   4 +
 t/t0012-help.sh                        |   2 +-
 t/t9902-completion.sh                  |   4 +-
 13 files changed, 249 insertions(+), 158 deletions(-)
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 15, 2018, 16:42 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v2 1/6] git.c: convert --list-builtins to --list-cmds=builtins

Even if this is a hidden option, let's make it a bit more generic since we're introducing more listing types.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  | 12 +++++++-----
 t/t0012-help.sh                        |  2 +-
 3 files changed, 9 insertions(+), 7 deletions(-)
Show changes to 3 files +9 −7

contrib/completion/git-completion.bash, git.c, t/t0012-help.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a757073945..3556838759 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -3049,7 +3049,7 @@ __git_complete_common () {
 __git_cmds_with_parseopt_helper=
 __git_support_parseopt_helper () {
 	test -n "$__git_cmds_with_parseopt_helper" ||
-		__git_cmds_with_parseopt_helper="$(__git --list-parseopt-builtins)"
+		__git_cmds_with_parseopt_helper="$(__git --list-cmds=parseopt)"
 
 	case " $__git_cmds_with_parseopt_helper " in
 	*" $1 "*)
diff --git a/git.c b/git.c
index 3a89893712..28bfa96d87 100644
--- a/git.c
+++ b/git.c
@@ -223,11 +223,13 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			}
 			(*argv)++;
 			(*argc)--;
-		} else if (!strcmp(cmd, "--list-builtins")) {
-			list_builtins(0, '\n');
-			exit(0);
-		} else if (!strcmp(cmd, "--list-parseopt-builtins")) {
-			list_builtins(NO_PARSEOPT, ' ');
+		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
+			if (!strcmp(cmd, "builtins"))
+				list_builtins(0, '\n');
+			else if (!strcmp(cmd, "parseopt"))
+				list_builtins(NO_PARSEOPT, ' ');
+			else
+				die("unsupported command listing type '%s'", cmd);
 			exit(0);
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 487b92a5de..fd2a7f27dc 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -50,7 +50,7 @@ test_expect_success "--help does not work for guides" "
 "
 
 test_expect_success 'generate builtin list' '
-	git --list-builtins >builtins
+	git --list-cmds=builtins >builtins
 '
 
 while read builtin
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 15, 2018, 16:42 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v2 4/6] git.c: implement --list-cmds=porcelain

This is useful for git-completion.bash because it needs this set of commands. Right now we have to maintain a separate command category in there.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash | 94 ++------------------------
 git.c                                  |  2 +
 help.c                                 | 12 ++++
 help.h                                 |  1 +
 t/t9902-completion.sh                  |  4 +-
 5 files changed, 20 insertions(+), 93 deletions(-)
Show changes to 5 files +20 −93

contrib/completion/git-completion.bash, git.c, help.c, help.h, t/t9902-completion.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a5f13ade20..9f17703aa7 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -839,14 +839,15 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git --list-cmds=all
+		git --list-cmds=$1
 	fi
 }
 
 __git_list_all_commands ()
 {
 	local i IFS=" "$'\n'
-	for i in $(__git_commands)
+	local category=${1-all}
+	for i in $(__git_commands $category)
 	do
 		case $i in
 		*--*)             : helper pattern;;
@@ -862,98 +863,11 @@ __git_compute_all_commands ()
 	__git_all_commands=$(__git_list_all_commands)
 }
 
-__git_list_porcelain_commands ()
-{
-	local i IFS=" "$'\n'
-	__git_compute_all_commands
-	for i in $__git_all_commands
-	do
-		case $i in
-		*--*)             : helper pattern;;
-		applymbox)        : ask gittus;;
-		applypatch)       : ask gittus;;
-		archimport)       : import;;
-		cat-file)         : plumbing;;
-		check-attr)       : plumbing;;
-		check-ignore)     : plumbing;;
-		check-mailmap)    : plumbing;;
-		check-ref-format) : plumbing;;
-		checkout-index)   : plumbing;;
-		column)           : internal helper;;
-		commit-tree)      : plumbing;;
-		count-objects)    : infrequent;;
-		credential)       : credentials;;
-		credential-*)     : credentials helper;;
-		cvsexportcommit)  : export;;
-		cvsimport)        : import;;
-		cvsserver)        : daemon;;
-		daemon)           : daemon;;
-		diff-files)       : plumbing;;
-		diff-index)       : plumbing;;
-		diff-tree)        : plumbing;;
-		fast-import)      : import;;
-		fast-export)      : export;;
-		fsck-objects)     : plumbing;;
-		fetch-pack)       : plumbing;;
-		fmt-merge-msg)    : plumbing;;
-		for-each-ref)     : plumbing;;
-		hash-object)      : plumbing;;
-		http-*)           : transport;;
-		index-pack)       : plumbing;;
-		init-db)          : deprecated;;
-		local-fetch)      : plumbing;;
-		ls-files)         : plumbing;;
-		ls-remote)        : plumbing;;
-		ls-tree)          : plumbing;;
-		mailinfo)         : plumbing;;
-		mailsplit)        : plumbing;;
-		merge-*)          : plumbing;;
-		mktree)           : plumbing;;
-		mktag)            : plumbing;;
-		pack-objects)     : plumbing;;
-		pack-redundant)   : plumbing;;
-		pack-refs)        : plumbing;;
-		parse-remote)     : plumbing;;
-		patch-id)         : plumbing;;
-		prune)            : plumbing;;
-		prune-packed)     : plumbing;;
-		quiltimport)      : import;;
-		read-tree)        : plumbing;;
-		receive-pack)     : plumbing;;
-		remote-*)         : transport;;
-		rerere)           : plumbing;;
-		rev-list)         : plumbing;;
-		rev-parse)        : plumbing;;
-		runstatus)        : plumbing;;
-		sh-setup)         : internal;;
-		shell)            : daemon;;
-		show-ref)         : plumbing;;
-		send-pack)        : plumbing;;
-		show-index)       : plumbing;;
-		ssh-*)            : transport;;
-		stripspace)       : plumbing;;
-		symbolic-ref)     : plumbing;;
-		unpack-file)      : plumbing;;
-		unpack-objects)   : plumbing;;
-		update-index)     : plumbing;;
-		update-ref)       : plumbing;;
-		update-server-info) : daemon;;
-		upload-archive)   : plumbing;;
-		upload-pack)      : plumbing;;
-		write-tree)       : plumbing;;
-		var)              : infrequent;;
-		verify-pack)      : infrequent;;
-		verify-tag)       : plumbing;;
-		*) echo $i;;
-		esac
-	done
-}
-
 __git_porcelain_commands=
 __git_compute_porcelain_commands ()
 {
 	test -n "$__git_porcelain_commands" ||
-	__git_porcelain_commands=$(__git_list_porcelain_commands)
+	__git_porcelain_commands=$(__git_list_all_commands porcelain)
 }
 
 # Lists all set config variables starting with the given section prefix,
diff --git a/git.c b/git.c
index 64f67e7f7f..153f56fb95 100644
--- a/git.c
+++ b/git.c
@@ -230,6 +230,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 				list_builtins(NO_PARSEOPT, ' ');
 			else if (!strcmp(cmd, "all"))
 				list_all_cmds();
+			else if (!strcmp(cmd, "porcelain"))
+				list_porcelain_cmds();
 			else
 				die("unsupported command listing type '%s'", cmd);
 			exit(0);
diff --git a/help.c b/help.c
index b5da7fa013..1523ca175c 100644
--- a/help.c
+++ b/help.c
@@ -272,6 +272,18 @@ void list_all_cmds(void)
 	clean_cmdnames(&other_cmds);
 }
 
+void list_porcelain_cmds(void)
+{
+	int i, nr = ARRAY_SIZE(command_list);
+	struct cmdname_help *cmds = command_list;
+
+	for (i = 0; i < nr; i++) {
+		if (cmds[i].category != CAT_mainporcelain)
+			continue;
+		puts(cmds[i].name);
+	}
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 0bf29f8dc5..33e2210ebd 100644
--- a/help.h
+++ b/help.h
@@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds(void);
+extern void list_porcelain_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 1b34caa1e1..4bfd26ddf9 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1208,8 +1208,6 @@ test_expect_success 'basic' '
 	grep -q "^add \$" out &&
 	# script
 	grep -q "^filter-branch \$" out &&
-	# plumbing
-	! grep -q "^ls-files \$" out &&
 
 	run_completion "git f" &&
 	! grep -q -v "^f" out
@@ -1272,7 +1270,7 @@ test_expect_success 'general options' '
 	test_completion "git --no-r" "--no-replace-objects "
 '
 
-test_expect_success 'general options plus command' '
+test_expect_failure 'general options plus command' '
 	test_completion "git --version check" "checkout " &&
 	test_completion "git --paginate check" "checkout " &&
 	test_completion "git --git-dir=foo check" "checkout " &&
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 15, 2018, 16:42 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

common-cmds.h is used to extract the list of common commands (by group) and a one-line summary of each command. Some information is dropped, for example command category or summary of other commands. Update generate-cmdlist.sh to keep all the information. The extra info will be used shortly.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 generate-cmdlist.sh | 61 +++++++++++++++++++++++++++++++++------------
 help.c              | 42 ++++++++++++++++++++++++++-----
 2 files changed, 81 insertions(+), 22 deletions(-)
Show changes to 2 files +81 −22

generate-cmdlist.sh, help.c

diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index eeea4b67ea..e0893e979a 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,17 +1,30 @@
 #!/bin/sh
 
+# Don't let locale affect this script.
+LC_ALL=C
+LANG=C
+export LC_ALL LANG
+
+command_list () {
+	sed '1,/^### command list/d;/^#/d' "$1"
+}
+
+category_list () {
+	command_list "$1" | awk '{print $2;}' | sort | uniq
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
-	char name[16];
+	char name[32];
 	char help[80];
-	unsigned char group;
+	unsigned int category;
+	unsigned int group;
 };
 
 static const char *common_cmd_groups[] = {"
 
 grps=grps$$.tmp
-match=match$$.tmp
-trap "rm -f '$grps' '$match'" 0 1 2 3 15
+trap "rm -f '$grps'" 0 1 2 3 15
 
 sed -n '
 	1,/^### common groups/b
@@ -23,28 +36,44 @@ sed -n '
 	' "$1"
 printf '};\n\n'
 
+echo "#define GROUP_NONE 0xff /* no common group */"
 n=0
-substnum=
 while read grp
 do
-	echo "^git-..*[ 	]$grp"
-	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
+	echo "#define GROUP_${grp:-NONE} $n"
 	n=$(($n+1))
-done <"$grps" >"$match"
+done <"$grps"
+echo
+
+echo '/*'
+printf 'static const char *cmd_categories[] = {\n'
+category_list "$1" |
+while read category; do
+	printf '\t\"'$category'\",\n'
+done
+printf '\tNULL\n};\n\n'
+echo '*/'
+
+n=0
+category_list "$1" |
+while read category; do
+	echo "#define CAT_$category $n"
+	n=$(($n+1))
+done
+echo
 
-printf 'static struct cmdname_help common_cmds[] = {\n'
-grep -f "$match" "$1" |
-sed 's/^git-//' |
+printf 'static struct cmdname_help command_list[] = {\n'
+command_list "$1" |
 sort |
-while read cmd tags
+while read cmd category tags
 do
-	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
+	name=${cmd/git-}
 	sed -n '
-		/^NAME/,/git-'"$cmd"'/H
+		/^NAME/,/'"$cmd"'/H
 		${
 			x
-			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
+			s/.*'"$cmd"' - \(.*\)/	{"'"$name"'", N_("\1"), CAT_'$category', GROUP_'${tags:-NONE}' },/
 			p
-		}' "Documentation/git-$cmd.txt"
+		}' "Documentation/$cmd.txt"
 done
 echo "};"
diff --git a/help.c b/help.c
index e155c39870..b5da7fa013 100644
--- a/help.c
+++ b/help.c
@@ -190,6 +190,27 @@ void list_commands(unsigned int colopts,
 	}
 }
 
+static void extract_common_cmds(struct cmdname_help **p_common_cmds,
+				int *p_nr)
+{
+	int i, nr = 0;
+	struct cmdname_help *common_cmds;
+
+	ALLOC_ARRAY(common_cmds, ARRAY_SIZE(command_list));
+
+	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
+		const struct cmdname_help *cmd = command_list + i;
+
+		if (cmd->category != CAT_mainporcelain)
+			continue;
+
+		common_cmds[nr++] = *cmd;
+	}
+
+	*p_common_cmds = common_cmds;
+	*p_nr = nr;
+}
+
 static int cmd_group_cmp(const void *elem1, const void *elem2)
 {
 	const struct cmdname_help *e1 = elem1;
@@ -206,17 +227,21 @@ void list_common_cmds_help(void)
 {
 	int i, longest = 0;
 	int current_grp = -1;
+	int nr = 0;
+	struct cmdname_help *common_cmds;
+
+	extract_common_cmds(&common_cmds, &nr);
 
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
+	for (i = 0; i < nr; i++) {
 		if (longest < strlen(common_cmds[i].name))
 			longest = strlen(common_cmds[i].name);
 	}
 
-	QSORT(common_cmds, ARRAY_SIZE(common_cmds), cmd_group_cmp);
+	QSORT(common_cmds, nr, cmd_group_cmp);
 
 	puts(_("These are common Git commands used in various situations:"));
 
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
+	for (i = 0; i < nr; i++) {
 		if (common_cmds[i].group != current_grp) {
 			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
 			current_grp = common_cmds[i].group;
@@ -226,6 +251,7 @@ void list_common_cmds_help(void)
 		mput_char(' ', longest - strlen(common_cmds[i].name));
 		puts(_(common_cmds[i].help));
 	}
+	free(common_cmds);
 }
 
 void list_all_cmds(void)
@@ -301,8 +327,9 @@ static const char bad_interpreter_advice[] =
 
 const char *help_unknown_cmd(const char *cmd)
 {
-	int i, n, best_similarity = 0;
+	int i, n, best_similarity = 0, nr_common;
 	struct cmdnames main_cmds, other_cmds;
+	struct cmdname_help *common_cmds;
 
 	memset(&main_cmds, 0, sizeof(main_cmds));
 	memset(&other_cmds, 0, sizeof(other_cmds));
@@ -317,6 +344,8 @@ const char *help_unknown_cmd(const char *cmd)
 	QSORT(main_cmds.names, main_cmds.cnt, cmdname_compare);
 	uniq(&main_cmds);
 
+	extract_common_cmds(&common_cmds, &nr_common);
+
 	/* This abuses cmdname->len for levenshtein distance */
 	for (i = 0, n = 0; i < main_cmds.cnt; i++) {
 		int cmp = 0; /* avoid compiler stupidity */
@@ -331,10 +360,10 @@ const char *help_unknown_cmd(const char *cmd)
 			die(_(bad_interpreter_advice), cmd, cmd);
 
 		/* Does the candidate appear in common_cmds list? */
-		while (n < ARRAY_SIZE(common_cmds) &&
+		while (n < nr_common &&
 		       (cmp = strcmp(common_cmds[n].name, candidate)) < 0)
 			n++;
-		if ((n < ARRAY_SIZE(common_cmds)) && !cmp) {
+		if ((n < nr_common) && !cmp) {
 			/* Yes, this is one of the common commands */
 			n++; /* use the entry from common_cmds[] */
 			if (starts_with(candidate, cmd)) {
@@ -347,6 +376,7 @@ const char *help_unknown_cmd(const char *cmd)
 		main_cmds.names[i]->len =
 			levenshtein(cmd, candidate, 0, 2, 1, 3) + 1;
 	}
+	FREE_AND_NULL(common_cmds);
 
 	QSORT(main_cmds.names, main_cmds.cnt, levenshtein_compare);
 
-- 
2.17.0.367.g5dd2e386c3
Junio C Hamano· Apr 16, 2018, 02:38 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
> +category_list () {
> +	command_list "$1" | awk '{print $2;}' | sort | uniq
> +}

Piping output of awk to sort/uniq, instead of processing all inside awk within the END block of the script, means that we are wasting two processes---I do not think we care too much about it, but some people might.

Øystein Walle· Apr 23, 2018, 08:23 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

Junio C Hamano <gitster@pobox.com> writes:
Show 11 quoted lines
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
> 
> > +category_list () {
> > +	command_list "$1" | awk '{print $2;}' | sort | uniq
> > +}
> 
> Piping output of awk to sort/uniq, instead of processing all inside
> awk within the END block of the script, means that we are wasting
> two processes---I do not think we care too much about it, but some
> people might.
> 
Can be written as:
    command_list "$1" | awk '!seen[$2]++ {print $2}'

This doesn't actually sort it, though, which I'm not sure whether is a good thing or a bad thing in this case. But it is less work, and being fast is nice for completion scripts.

Øsse
SZEDER Gábor· Apr 23, 2018, 09:59 UTC · re: Øystein Walle · lore

Re: [PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

Show 21 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
> > 
> > > +category_list () {
> > > +	command_list "$1" | awk '{print $2;}' | sort | uniq
> > > +}
> > 
> > Piping output of awk to sort/uniq, instead of processing all inside
> > awk within the END block of the script, means that we are wasting
> > two processes---I do not think we care too much about it, but some
> > people might.
> > 
> 
> Can be written as:
> 
>     command_list "$1" | awk '!seen[$2]++ {print $2}'
> 
> This doesn't actually sort it, though, which I'm not sure whether is a
> good thing or a bad thing in this case. But it is less work, and being fast is
> nice for completion scripts.
This script is run during the build process, not during completion.

(The order wouldn't matter for completion, because the shell would sort possible completion words anyway.)

Junio C Hamano· Apr 16, 2018, 06:28 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 5 quoted lines
> @@ -23,28 +36,44 @@ sed -n '
>  	' "$1"
>  printf '};\n\n'
>  
> +echo "#define GROUP_NONE 0xff /* no common group */"

Some later code forgets about this value, and causes "git<ENTER>" to segfault at the end of this entire series.

Namely, here:
Show 5 quoted lines
> -	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
> +	for (i = 0; i < nr; i++) {
>  		if (common_cmds[i].group != current_grp) {
>  			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
>  			current_grp = common_cmds[i].group;
where common_cmd_groups[] gets overrun.

Here is a squash I'll queue on top to keep the tip of 'pu' at least buildable.

 help.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Show changes to help.c +3 −1
diff --git a/help.c b/help.c
index a44f4a113e..74591d5ebc 100644
--- a/help.c
+++ b/help.c
@@ -242,7 +242,9 @@ void list_common_cmds_help(void)
 	puts(_("These are common Git commands used in various situations:"));
 
 	for (i = 0; i < nr; i++) {
-		if (common_cmds[i].group != current_grp) {
+		if (ARRAY_SIZE(common_cmd_groups) <= common_cmds[i].group)
+			; /* skip */
+		else if (common_cmds[i].group != current_grp) {
 			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
 			current_grp = common_cmds[i].group;
 		}
Duy Nguyen· Apr 16, 2018, 15:43 UTC · re: Junio C Hamano · lore

Re: [PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

On Mon, Apr 16, 2018 at 8:28 AM, Junio C Hamano <gitster@pobox.com> wrote:
Show 20 quoted lines
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>
>> @@ -23,28 +36,44 @@ sed -n '
>>       ' "$1"
>>  printf '};\n\n'
>>
>> +echo "#define GROUP_NONE 0xff /* no common group */"
>
> Some later code forgets about this value, and causes "git<ENTER>" to
> segfault at the end of this entire series.
>
> Namely, here:
>
>> -     for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
>> +     for (i = 0; i < nr; i++) {
>>               if (common_cmds[i].group != current_grp) {
>>                       printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
>>                       current_grp = common_cmds[i].group;
>
> where common_cmd_groups[] gets overrun.
Argh!! I thought I tested everything. Sorry for the sloppy quality.
>
> Here is a squash I'll queue on top to keep the tip of 'pu' at least
> buildable.

Thanks. It's actually interesting that we have main porcelain commands that belong to no group. I'll try to classify them so that they show up as well.

-- 
Duy
SZEDER Gábor· Apr 16, 2018, 15:43 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

On Sun, Apr 15, 2018 at 6:42 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
Show 16 quoted lines
> common-cmds.h is used to extract the list of common commands (by
> group) and a one-line summary of each command. Some information is
> dropped, for example command category or summary of other commands.
> Update generate-cmdlist.sh to keep all the information. The extra info
> will be used shortly.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  generate-cmdlist.sh | 61 +++++++++++++++++++++++++++++++++------------
>  help.c              | 42 ++++++++++++++++++++++++++-----
>  2 files changed, 81 insertions(+), 22 deletions(-)
>
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index eeea4b67ea..e0893e979a 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
Show 11 quoted lines
> -printf 'static struct cmdname_help common_cmds[] = {\n'
> -grep -f "$match" "$1" |
> -sed 's/^git-//' |
> +printf 'static struct cmdname_help command_list[] = {\n'
> +command_list "$1" |
>  sort |
> -while read cmd tags
> +while read cmd category tags
>  do
> -       tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
> +       name=${cmd/git-}
There are two issues with this line:
- This is a "regular" shell script, therefore it must not use pattern
  substitution.
- The pattern substitution would remove the string "git-" in the middle of
  the variable as well; I suspect this is undesired.

I think that the remove matching prefix pattern substitution should be used here.

Ramsay Jones· Apr 16, 2018, 16:25 UTC · re: SZEDER Gábor · lore

Re: [PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

On 16/04/18 16:43, SZEDER Gábor wrote:
Show 40 quoted lines
> On Sun, Apr 15, 2018 at 6:42 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
>> common-cmds.h is used to extract the list of common commands (by
>> group) and a one-line summary of each command. Some information is
>> dropped, for example command category or summary of other commands.
>> Update generate-cmdlist.sh to keep all the information. The extra info
>> will be used shortly.
>>
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> ---
>>  generate-cmdlist.sh | 61 +++++++++++++++++++++++++++++++++------------
>>  help.c              | 42 ++++++++++++++++++++++++++-----
>>  2 files changed, 81 insertions(+), 22 deletions(-)
>>
>> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
>> index eeea4b67ea..e0893e979a 100755
>> --- a/generate-cmdlist.sh
>> +++ b/generate-cmdlist.sh
> 
>> -printf 'static struct cmdname_help common_cmds[] = {\n'
>> -grep -f "$match" "$1" |
>> -sed 's/^git-//' |
>> +printf 'static struct cmdname_help command_list[] = {\n'
>> +command_list "$1" |
>>  sort |
>> -while read cmd tags
>> +while read cmd category tags
>>  do
>> -       tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
>> +       name=${cmd/git-}
> 
> There are two issues with this line:
> 
> - This is a "regular" shell script, therefore it must not use pattern
>   substitution.
> 
> - The pattern substitution would remove the string "git-" in the middle of
>   the variable as well; I suspect this is undesired.
> 
> I think that the remove matching prefix pattern substitution should be
> used here.

This also, apparently, doesn't work with dash. On cygwin, which has bash as /bin/sh, this builds common-cmds.h just fine. On linux, which has dash as /bin/sh, this fails, leaving a truncated file:

  $ bash generate-cmdlist.sh command-list.txt >zzz
  $ tail -3 zzz
  	{"gitweb", N_("Git web interface (web frontend to Git repositories)"), CAT_ancillaryinterrogators, GROUP_NONE },
  	{"workflows", N_("An overview of recommended workflows with Git"), CAT_guide, GROUP_NONE },
  };
  $ dash generate-cmdlist.sh command-list.txt >zzz
  generate-cmdlist.sh: 73: generate-cmdlist.sh: Bad substitution
  $ tail -3 zzz
  static struct cmdname_help command_list[] = {
  };
  $ 
This leads to a very broken 'git help'.

ATB, Ramsay Jones

Junio C Hamano· Apr 16, 2018, 22:42 UTC · re: SZEDER Gábor · lore

Re: [PATCH v2 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

SZEDER Gábor <szeder.dev@gmail.com> writes:
Show 9 quoted lines
>> +while read cmd category tags
>>  do
>> -       tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
>> +       name=${cmd/git-}
>
> There are two issues with this line:
>
> - This is a "regular" shell script, therefore it must not use pattern
>   substitution.
Oops.  I missed that.  Thanks.
Show 6 quoted lines
>
> - The pattern substitution would remove the string "git-" in the middle of
>   the variable as well; I suspect this is undesired.
>
> I think that the remove matching prefix pattern substitution should be
> used here.
Nguyễn Thái Ngọc Duy· Apr 15, 2018, 16:42 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v2 6/6] help: use command-list.txt for the source of guides

The help command currently hard codes the list of guides and their summary in C. Let's move this list to command-list.txt. This lets us extract summary lines from Documentation/git*.txt. This also potentially lets us lists guides in git.txt, but I'll leave that for now.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/gitattributes.txt |  2 +-
 Documentation/gitmodules.txt    |  2 +-
 Documentation/gitrevisions.txt  |  2 +-
 builtin/help.c                  | 32 --------------------------------
 command-list.txt                |  8 ++++++++
 generate-cmdlist.sh             |  6 +++++-
 help.c                          | 22 ++++++++++++++++++++++
 help.h                          |  1 +
 8 files changed, 39 insertions(+), 36 deletions(-)
Show changes to 8 files +39 −36

Documentation/gitattributes.txt, Documentation/gitmodules.txt, Documentation/gitrevisions.txt, builtin/help.c, command-list.txt, generate-cmdlist.sh, help.c, help.h

diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 1094fe2b5b..083c2f380d 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -3,7 +3,7 @@ gitattributes(5)
 
 NAME
 ----
-gitattributes - defining attributes per path
+gitattributes - Defining attributes per path
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index db5d47eb19..4d63def206 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -3,7 +3,7 @@ gitmodules(5)
 
 NAME
 ----
-gitmodules - defining submodule properties
+gitmodules - Defining submodule properties
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitrevisions.txt b/Documentation/gitrevisions.txt
index 27dec5b91d..1f6cceaefb 100644
--- a/Documentation/gitrevisions.txt
+++ b/Documentation/gitrevisions.txt
@@ -3,7 +3,7 @@ gitrevisions(7)
 
 NAME
 ----
-gitrevisions - specifying revisions and ranges for Git
+gitrevisions - Specifying revisions and ranges for Git
 
 SYNOPSIS
 --------
diff --git a/builtin/help.c b/builtin/help.c
index 0e0af8426a..5727fb5e51 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -402,38 +402,6 @@ static void show_html_page(const char *git_cmd)
 	open_html(page_path.buf);
 }
 
-static struct {
-	const char *name;
-	const char *help;
-} common_guides[] = {
-	{ "attributes", N_("Defining attributes per path") },
-	{ "everyday", N_("Everyday Git With 20 Commands Or So") },
-	{ "glossary", N_("A Git glossary") },
-	{ "ignore", N_("Specifies intentionally untracked files to ignore") },
-	{ "modules", N_("Defining submodule properties") },
-	{ "revisions", N_("Specifying revisions and ranges for Git") },
-	{ "tutorial", N_("A tutorial introduction to Git (for version 1.5.1 or newer)") },
-	{ "workflows", N_("An overview of recommended workflows with Git") },
-};
-
-static void list_common_guides_help(void)
-{
-	int i, longest = 0;
-
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		if (longest < strlen(common_guides[i].name))
-			longest = strlen(common_guides[i].name);
-	}
-
-	puts(_("The common Git guides are:\n"));
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		printf("   %s   ", common_guides[i].name);
-		mput_char(' ', longest - strlen(common_guides[i].name));
-		puts(_(common_guides[i].help));
-	}
-	putchar('\n');
-}
-
 static const char *check_git_cmd(const char* cmd)
 {
 	char *alias;
diff --git a/command-list.txt b/command-list.txt
index a1fad28fd8..0809a19184 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -149,3 +149,11 @@ gitweb                                  ancillaryinterrogators
 git-whatchanged                         ancillaryinterrogators
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
+gitattributes                           guide
+giteveryday                             guide
+gitglossary                             guide
+gitignore                               guide
+gitmodules                              guide
+gitrevisions                            guide
+gittutorial                             guide
+gitworkflows                            guide
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index e0893e979a..e35f3e357b 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -67,7 +67,11 @@ command_list "$1" |
 sort |
 while read cmd category tags
 do
-	name=${cmd/git-}
+	if [ "$category" = guide ]; then
+		name=${cmd/git}
+	else
+		name=${cmd/git-}
+	fi
 	sed -n '
 		/^NAME/,/'"$cmd"'/H
 		${
diff --git a/help.c b/help.c
index 7f72051641..a44f4a113e 100644
--- a/help.c
+++ b/help.c
@@ -313,6 +313,28 @@ static void list_commands_by_category(int cat, struct cmdname_help *cmds,
 	}
 }
 
+void list_common_guides_help(void)
+{
+	int i, longest = 0;
+	int nr = ARRAY_SIZE(command_list);
+	struct cmdname_help *cmds = command_list;
+
+	QSORT(cmds, nr, cmd_category_cmp);
+
+	for (i = 0; i < nr; i++) {
+		struct cmdname_help *cmd = cmds + i;
+
+		if (cmd->category != CAT_guide)
+			continue;
+		if (longest < strlen(cmd->name))
+			longest = strlen(cmd->name);
+	}
+
+	puts(_("The common Git guides are:\n"));
+	list_commands_by_category(CAT_guide, cmds, nr, longest);
+	putchar('\n');
+}
+
 void list_all_cmds_help(void)
 {
 	int i, longest = 0;
diff --git a/help.h b/help.h
index 62449f1b7e..de094e9e65 100644
--- a/help.h
+++ b/help.h
@@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds_help(void);
+extern void list_common_guides_help(void);
 extern void list_all_cmds(void);
 extern void list_porcelain_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 15, 2018, 16:42 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v2 2/6] git.c: implement --list-cmds=all and use it in git-completion.bash

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  |  2 ++
 help.c                                 | 18 ++++++++++++++++++
 help.h                                 |  1 +
 4 files changed, 22 insertions(+), 1 deletion(-)
Show changes to 4 files +22 −1

contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3556838759..a5f13ade20 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -839,7 +839,7 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git help -a|egrep '^  [a-zA-Z0-9]'
+		git --list-cmds=all
 	fi
 }
 
diff --git a/git.c b/git.c
index 28bfa96d87..64f67e7f7f 100644
--- a/git.c
+++ b/git.c
@@ -228,6 +228,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 				list_builtins(0, '\n');
 			else if (!strcmp(cmd, "parseopt"))
 				list_builtins(NO_PARSEOPT, ' ');
+			else if (!strcmp(cmd, "all"))
+				list_all_cmds();
 			else
 				die("unsupported command listing type '%s'", cmd);
 			exit(0);
diff --git a/help.c b/help.c
index 60071a9bea..e155c39870 100644
--- a/help.c
+++ b/help.c
@@ -228,6 +228,24 @@ void list_common_cmds_help(void)
 	}
 }
 
+void list_all_cmds(void)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < main_cmds.cnt; i++)
+		puts(main_cmds.names[i]->name);
+	for (i = 0; i < other_cmds.cnt; i++)
+		puts(other_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index b21d7c94e8..0bf29f8dc5 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.367.g5dd2e386c3
Junio C Hamano· Apr 16, 2018, 02:30 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v2 2/6] git.c: implement --list-cmds=all and use it in git-completion.bash

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 20 quoted lines
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  contrib/completion/git-completion.bash |  2 +-
>  git.c                                  |  2 ++
>  help.c                                 | 18 ++++++++++++++++++
>  help.h                                 |  1 +
>  4 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index 3556838759..a5f13ade20 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -839,7 +839,7 @@ __git_commands () {
>  	then
>  		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
>  	else
> -		git help -a|egrep '^  [a-zA-Z0-9]'
> +		git --list-cmds=all
>  	fi
>  }

To those of us who install a copy of git-completion.bash somewhere in $HOME and forget about it, while installing different versions of Git all the time for testing, may see breakage caused by an invalid combination of having new completion script with Git that does not know about --list=cmds=<all> option. I do not think it matters too much, though ;-)

Show 18 quoted lines
> +void list_all_cmds(void)
> +{
> +	struct cmdnames main_cmds, other_cmds;
> +	int i;
> +
> +	memset(&main_cmds, 0, sizeof(main_cmds));
> +	memset(&other_cmds, 0, sizeof(other_cmds));
> +	load_command_list("git-", &main_cmds, &other_cmds);
> +
> +	for (i = 0; i < main_cmds.cnt; i++)
> +		puts(main_cmds.names[i]->name);
> +	for (i = 0; i < other_cmds.cnt; i++)
> +		puts(other_cmds.names[i]->name);
> +
> +	clean_cmdnames(&main_cmds);
> +	clean_cmdnames(&other_cmds);
> +}
> +
OK.

By reusing load_command_list(), the duplicate-removal logic at its end that is used for "help -a" kicks in; the above looks good.

Nguyễn Thái Ngọc Duy· Apr 15, 2018, 16:42 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v2 5/6] help: add "-a --verbose" to list all commands with synopsis

This lists all recognized commands [1] by category. The group order follows closely git.txt.

[1] We may actually show commands that are not built (e.g. if you set NO_PERL you don't have git-instaweb but it's still listed here). I ignore the problem because on Linux a git package could be split anyway. The "git-core" package may not contain git-instaweb even if it's built because it may end up in a separate package. We can't know anyway.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-help.txt |  4 ++-
 builtin/help.c             |  7 ++++
 help.c                     | 69 ++++++++++++++++++++++++++++++++++++++
 help.h                     |  1 +
 4 files changed, 80 insertions(+), 1 deletion(-)
Show changes to 4 files +80 −1

Documentation/git-help.txt, builtin/help.c, help.c, help.h

diff --git a/Documentation/git-help.txt b/Documentation/git-help.txt
index 40d328a4b3..a40fc38d8b 100644
--- a/Documentation/git-help.txt
+++ b/Documentation/git-help.txt
@@ -8,7 +8,7 @@ git-help - Display help information about Git
 SYNOPSIS
 --------
 [verse]
-'git help' [-a|--all] [-g|--guide]
+'git help' [-a|--all [--verbose]] [-g|--guide]
 	   [-i|--info|-m|--man|-w|--web] [COMMAND|GUIDE]
 
 DESCRIPTION
@@ -42,6 +42,8 @@ OPTIONS
 --all::
 	Prints all the available commands on the standard output. This
 	option overrides any given command or guide name.
+	When used with `--verbose` print description for all recognized
+	commands.
 
 -g::
 --guides::
diff --git a/builtin/help.c b/builtin/help.c
index 598867cfea..0e0af8426a 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -36,6 +36,7 @@ static const char *html_path;
 
 static int show_all = 0;
 static int show_guides = 0;
+static int verbose;
 static unsigned int colopts;
 static enum help_format help_format = HELP_FORMAT_NONE;
 static int exclude_guides;
@@ -48,6 +49,7 @@ static struct option builtin_help_options[] = {
 			HELP_FORMAT_WEB),
 	OPT_SET_INT('i', "info", &help_format, N_("show info page"),
 			HELP_FORMAT_INFO),
+	OPT__VERBOSE(&verbose, N_("print command description")),
 	OPT_END(),
 };
 
@@ -463,6 +465,11 @@ int cmd_help(int argc, const char **argv, const char *prefix)
 
 	if (show_all) {
 		git_config(git_help_config, NULL);
+		if (verbose) {
+			setup_pager();
+			list_all_cmds_help();
+			return 0;
+		}
 		printf(_("usage: %s%s"), _(git_usage_string), "\n\n");
 		load_command_list("git-", &main_cmds, &other_cmds);
 		list_commands(colopts, &main_cmds, &other_cmds);
diff --git a/help.c b/help.c
index 1523ca175c..7f72051641 100644
--- a/help.c
+++ b/help.c
@@ -284,6 +284,75 @@ void list_porcelain_cmds(void)
 	}
 }
 
+static int cmd_category_cmp(const void *elem1, const void *elem2)
+{
+	const struct cmdname_help *e1 = elem1;
+	const struct cmdname_help *e2 = elem2;
+
+	if (e1->category < e2->category)
+		return -1;
+	if (e1->category > e2->category)
+		return 1;
+	return strcmp(e1->name, e2->name);
+}
+
+static void list_commands_by_category(int cat, struct cmdname_help *cmds,
+				      int nr, int longest)
+{
+	int i;
+
+	for (i = 0; i < nr; i++) {
+		struct cmdname_help *cmd = cmds + i;
+
+		if (cmd->category != cat)
+			continue;
+
+		printf("   %s   ", cmd->name);
+		mput_char(' ', longest - strlen(cmd->name));
+		puts(_(cmd->help));
+	}
+}
+
+void list_all_cmds_help(void)
+{
+	int i, longest = 0;
+	int nr = ARRAY_SIZE(command_list);
+	struct cmdname_help *cmds = command_list;
+
+	for (i = 0; i < nr; i++) {
+		struct cmdname_help *cmd = cmds + i;
+
+		if (longest < strlen(cmd->name))
+			longest = strlen(cmd->name);
+	}
+
+	QSORT(cmds, nr, cmd_category_cmp);
+
+	printf("%s\n\n", _("Main Porcelain Commands"));
+	list_commands_by_category(CAT_mainporcelain, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Ancillary Commands / Manipulators"));
+	list_commands_by_category(CAT_ancillarymanipulators, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Ancillary Commands / Interrogators"));
+	list_commands_by_category(CAT_ancillaryinterrogators, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Interacting with Others"));
+	list_commands_by_category(CAT_foreignscminterface, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Low-level Commands / Manipulators"));
+	list_commands_by_category(CAT_plumbingmanipulators, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Low-level Commands / Interrogators"));
+	list_commands_by_category(CAT_plumbinginterrogators, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Low-level Commands / Synching Repositories"));
+	list_commands_by_category(CAT_synchingrepositories, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Low-level Commands / Internal Helpers"));
+	list_commands_by_category(CAT_purehelpers, cmds, nr, longest);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 33e2210ebd..62449f1b7e 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds_help(void);
 extern void list_all_cmds(void);
 extern void list_porcelain_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 21, 2018, 16:54 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v3 0/6] Keep all info in command-list.txt in git binary

Changes:
- remove the deprecated column in command-list.txt. My change break it
  anyway if anyone uses it.
- fix up failed tests that I marked in the RFC and kinda forgot about it.
- fix bashisms in generate-cmdlist.sh
- fix segfaul in "git help"
Nguyễn Thái Ngọc Duy (6):
  git.c: convert --list-*builtins to --list-cmds=*
  git.c: implement --list-cmds=all and use it in git-completion.bash
  generate-cmdlist.sh: keep all information in common-cmds.h
  git.c: implement --list-cmds=porcelain
  help: add "-a --verbose" to list all commands with synopsis
  help: use command-list.txt for the source of guides
 Documentation/git-help.txt             |   4 +-
 Documentation/gitattributes.txt        |   2 +-
 Documentation/gitmodules.txt           |   2 +-
 Documentation/gitrevisions.txt         |   2 +-
 builtin/help.c                         |  39 ++----
 command-list.txt                       |  10 +-
 contrib/completion/git-completion.bash | 108 ++--------------
 generate-cmdlist.sh                    |  57 ++++++---
 git.c                                  |  16 ++-
 help.c                                 | 164 ++++++++++++++++++++++++-
 help.h                                 |   4 +
 t/t0012-help.sh                        |  11 +-
 t/t9902-completion.sh                  |   6 +-
 13 files changed, 263 insertions(+), 162 deletions(-)
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 21, 2018, 16:54 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v3 1/6] git.c: convert --list-*builtins to --list-cmds=*

Even if these are hidden options, let's make them a bit more generic since we're introducing more listing types shortly.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  | 12 +++++++-----
 t/t0012-help.sh                        |  2 +-
 3 files changed, 9 insertions(+), 7 deletions(-)
Show changes to 3 files +9 −7

contrib/completion/git-completion.bash, git.c, t/t0012-help.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a757073945..3556838759 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -3049,7 +3049,7 @@ __git_complete_common () {
 __git_cmds_with_parseopt_helper=
 __git_support_parseopt_helper () {
 	test -n "$__git_cmds_with_parseopt_helper" ||
-		__git_cmds_with_parseopt_helper="$(__git --list-parseopt-builtins)"
+		__git_cmds_with_parseopt_helper="$(__git --list-cmds=parseopt)"
 
 	case " $__git_cmds_with_parseopt_helper " in
 	*" $1 "*)
diff --git a/git.c b/git.c
index 3a89893712..28bfa96d87 100644
--- a/git.c
+++ b/git.c
@@ -223,11 +223,13 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			}
 			(*argv)++;
 			(*argc)--;
-		} else if (!strcmp(cmd, "--list-builtins")) {
-			list_builtins(0, '\n');
-			exit(0);
-		} else if (!strcmp(cmd, "--list-parseopt-builtins")) {
-			list_builtins(NO_PARSEOPT, ' ');
+		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
+			if (!strcmp(cmd, "builtins"))
+				list_builtins(0, '\n');
+			else if (!strcmp(cmd, "parseopt"))
+				list_builtins(NO_PARSEOPT, ' ');
+			else
+				die("unsupported command listing type '%s'", cmd);
 			exit(0);
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 487b92a5de..fd2a7f27dc 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -50,7 +50,7 @@ test_expect_success "--help does not work for guides" "
 "
 
 test_expect_success 'generate builtin list' '
-	git --list-builtins >builtins
+	git --list-cmds=builtins >builtins
 '
 
 while read builtin
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 21, 2018, 16:54 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v3 2/6] git.c: implement --list-cmds=all and use it in git-completion.bash

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  |  2 ++
 help.c                                 | 18 ++++++++++++++++++
 help.h                                 |  1 +
 4 files changed, 22 insertions(+), 1 deletion(-)
Show changes to 4 files +22 −1

contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3556838759..a5f13ade20 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -839,7 +839,7 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git help -a|egrep '^  [a-zA-Z0-9]'
+		git --list-cmds=all
 	fi
 }
 
diff --git a/git.c b/git.c
index 28bfa96d87..64f67e7f7f 100644
--- a/git.c
+++ b/git.c
@@ -228,6 +228,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 				list_builtins(0, '\n');
 			else if (!strcmp(cmd, "parseopt"))
 				list_builtins(NO_PARSEOPT, ' ');
+			else if (!strcmp(cmd, "all"))
+				list_all_cmds();
 			else
 				die("unsupported command listing type '%s'", cmd);
 			exit(0);
diff --git a/help.c b/help.c
index 60071a9bea..e155c39870 100644
--- a/help.c
+++ b/help.c
@@ -228,6 +228,24 @@ void list_common_cmds_help(void)
 	}
 }
 
+void list_all_cmds(void)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < main_cmds.cnt; i++)
+		puts(main_cmds.names[i]->name);
+	for (i = 0; i < other_cmds.cnt; i++)
+		puts(other_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index b21d7c94e8..0bf29f8dc5 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.367.g5dd2e386c3
SZEDER Gábor· Apr 23, 2018, 11:20 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v3 2/6] git.c: implement --list-cmds=all and use it in git-completion.bash

On Sat, Apr 21, 2018 at 6:54 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>

Oh, look, an empty commit message! :) So what does this option do and why use it in the completion script?

Nguyễn Thái Ngọc Duy· Apr 21, 2018, 16:54 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v3 3/6] generate-cmdlist.sh: keep all information in common-cmds.h

common-cmds.h is used to extract the list of common commands (by group) and a one-line summary of each command. Some information is dropped, for example command category or summary of other commands. Update generate-cmdlist.sh to keep all the information. The extra info will be used shortly.

The "deprecated" column is dropped from command-list.txt since it's not really used and the new parsing code can't deal with it. When we do need deprecated attribute, we could add it back and adjust generate-cmdlist.sh at the same time.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 command-list.txt    |  2 +-
 generate-cmdlist.sh | 53 +++++++++++++++++++++++++++++++--------------
 help.c              | 43 +++++++++++++++++++++++++++++++-----
 3 files changed, 75 insertions(+), 23 deletions(-)
Show changes to 3 files +75 −23

command-list.txt, generate-cmdlist.sh, help.c

diff --git a/command-list.txt b/command-list.txt
index a1fad28fd8..2c81d8db74 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -9,7 +9,7 @@ history      grow, mark and tweak your common history
 remote       collaborate (see also: git help workflows)
 
 ### command list (do not change this line)
-# command name                          category [deprecated] [common]
+# command name                          category                [common]
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index eeea4b67ea..05722b1392 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,17 +1,30 @@
 #!/bin/sh
 
+# Don't let locale affect this script.
+LC_ALL=C
+LANG=C
+export LC_ALL LANG
+
+command_list () {
+	sed '1,/^### command list/d;/^#/d' "$1"
+}
+
+category_list () {
+	command_list "$1" | awk '{print $2;}' | sort | uniq
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
-	char name[16];
+	char name[32];
 	char help[80];
-	unsigned char group;
+	unsigned int category;
+	unsigned int group;
 };
 
 static const char *common_cmd_groups[] = {"
 
 grps=grps$$.tmp
-match=match$$.tmp
-trap "rm -f '$grps' '$match'" 0 1 2 3 15
+trap "rm -f '$grps'" 0 1 2 3 15
 
 sed -n '
 	1,/^### common groups/b
@@ -23,28 +36,36 @@ sed -n '
 	' "$1"
 printf '};\n\n'
 
+echo "#define GROUP_NONE 0xff"
 n=0
-substnum=
 while read grp
 do
-	echo "^git-..*[ 	]$grp"
-	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
+	echo "#define GROUP_${grp:-NONE} $n"
 	n=$(($n+1))
-done <"$grps" >"$match"
+done <"$grps"
+echo
+
+n=0
+category_list "$1" |
+while read category; do
+	echo "#define CAT_$category $n"
+	n=$(($n+1))
+done
+echo
 
-printf 'static struct cmdname_help common_cmds[] = {\n'
-grep -f "$match" "$1" |
-sed 's/^git-//' |
+printf 'static struct cmdname_help command_list[] = {\n'
+command_list "$1" |
 sort |
-while read cmd tags
+while read cmd category tags
 do
-	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
+	prefix=git-
+	name=$(echo $cmd | sed "s/^${prefix}//")
 	sed -n '
-		/^NAME/,/git-'"$cmd"'/H
+		/^NAME/,/'"$cmd"'/H
 		${
 			x
-			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
+			s/.*'"$cmd"' - \(.*\)/	{"'"$name"'", N_("\1"), CAT_'$category', GROUP_'${tags:-NONE}' },/
 			p
-		}' "Documentation/git-$cmd.txt"
+		}' "Documentation/$cmd.txt"
 done
 echo "};"
diff --git a/help.c b/help.c
index e155c39870..e63006c333 100644
--- a/help.c
+++ b/help.c
@@ -190,6 +190,28 @@ void list_commands(unsigned int colopts,
 	}
 }
 
+static void extract_common_cmds(struct cmdname_help **p_common_cmds,
+				int *p_nr)
+{
+	int i, nr = 0;
+	struct cmdname_help *common_cmds;
+
+	ALLOC_ARRAY(common_cmds, ARRAY_SIZE(command_list));
+
+	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
+		const struct cmdname_help *cmd = command_list + i;
+
+		if (cmd->category != CAT_mainporcelain ||
+		    cmd->group == GROUP_NONE)
+			continue;
+
+		common_cmds[nr++] = *cmd;
+	}
+
+	*p_common_cmds = common_cmds;
+	*p_nr = nr;
+}
+
 static int cmd_group_cmp(const void *elem1, const void *elem2)
 {
 	const struct cmdname_help *e1 = elem1;
@@ -206,17 +228,21 @@ void list_common_cmds_help(void)
 {
 	int i, longest = 0;
 	int current_grp = -1;
+	int nr = 0;
+	struct cmdname_help *common_cmds;
+
+	extract_common_cmds(&common_cmds, &nr);
 
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
+	for (i = 0; i < nr; i++) {
 		if (longest < strlen(common_cmds[i].name))
 			longest = strlen(common_cmds[i].name);
 	}
 
-	QSORT(common_cmds, ARRAY_SIZE(common_cmds), cmd_group_cmp);
+	QSORT(common_cmds, nr, cmd_group_cmp);
 
 	puts(_("These are common Git commands used in various situations:"));
 
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
+	for (i = 0; i < nr; i++) {
 		if (common_cmds[i].group != current_grp) {
 			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
 			current_grp = common_cmds[i].group;
@@ -226,6 +252,7 @@ void list_common_cmds_help(void)
 		mput_char(' ', longest - strlen(common_cmds[i].name));
 		puts(_(common_cmds[i].help));
 	}
+	free(common_cmds);
 }
 
 void list_all_cmds(void)
@@ -301,8 +328,9 @@ static const char bad_interpreter_advice[] =
 
 const char *help_unknown_cmd(const char *cmd)
 {
-	int i, n, best_similarity = 0;
+	int i, n, best_similarity = 0, nr_common;
 	struct cmdnames main_cmds, other_cmds;
+	struct cmdname_help *common_cmds;
 
 	memset(&main_cmds, 0, sizeof(main_cmds));
 	memset(&other_cmds, 0, sizeof(other_cmds));
@@ -317,6 +345,8 @@ const char *help_unknown_cmd(const char *cmd)
 	QSORT(main_cmds.names, main_cmds.cnt, cmdname_compare);
 	uniq(&main_cmds);
 
+	extract_common_cmds(&common_cmds, &nr_common);
+
 	/* This abuses cmdname->len for levenshtein distance */
 	for (i = 0, n = 0; i < main_cmds.cnt; i++) {
 		int cmp = 0; /* avoid compiler stupidity */
@@ -331,10 +361,10 @@ const char *help_unknown_cmd(const char *cmd)
 			die(_(bad_interpreter_advice), cmd, cmd);
 
 		/* Does the candidate appear in common_cmds list? */
-		while (n < ARRAY_SIZE(common_cmds) &&
+		while (n < nr_common &&
 		       (cmp = strcmp(common_cmds[n].name, candidate)) < 0)
 			n++;
-		if ((n < ARRAY_SIZE(common_cmds)) && !cmp) {
+		if ((n < nr_common) && !cmp) {
 			/* Yes, this is one of the common commands */
 			n++; /* use the entry from common_cmds[] */
 			if (starts_with(candidate, cmd)) {
@@ -347,6 +377,7 @@ const char *help_unknown_cmd(const char *cmd)
 		main_cmds.names[i]->len =
 			levenshtein(cmd, candidate, 0, 2, 1, 3) + 1;
 	}
+	FREE_AND_NULL(common_cmds);
 
 	QSORT(main_cmds.names, main_cmds.cnt, levenshtein_compare);
 
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 21, 2018, 16:54 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v3 6/6] help: use command-list.txt for the source of guides

The help command currently hard codes the list of guides and their summary in C. Let's move this list to command-list.txt. This lets us extract summary lines from Documentation/git*.txt. This also potentially lets us lists guides in git.txt, but I'll leave that for now.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/gitattributes.txt |  2 +-
 Documentation/gitmodules.txt    |  2 +-
 Documentation/gitrevisions.txt  |  2 +-
 builtin/help.c                  | 32 --------------------------------
 command-list.txt                |  8 ++++++++
 generate-cmdlist.sh             |  6 +++++-
 help.c                          | 22 ++++++++++++++++++++++
 help.h                          |  1 +
 8 files changed, 39 insertions(+), 36 deletions(-)
Show changes to 8 files +39 −36

Documentation/gitattributes.txt, Documentation/gitmodules.txt, Documentation/gitrevisions.txt, builtin/help.c, command-list.txt, generate-cmdlist.sh, help.c, help.h

diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 1094fe2b5b..083c2f380d 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -3,7 +3,7 @@ gitattributes(5)
 
 NAME
 ----
-gitattributes - defining attributes per path
+gitattributes - Defining attributes per path
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index db5d47eb19..4d63def206 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -3,7 +3,7 @@ gitmodules(5)
 
 NAME
 ----
-gitmodules - defining submodule properties
+gitmodules - Defining submodule properties
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitrevisions.txt b/Documentation/gitrevisions.txt
index 27dec5b91d..1f6cceaefb 100644
--- a/Documentation/gitrevisions.txt
+++ b/Documentation/gitrevisions.txt
@@ -3,7 +3,7 @@ gitrevisions(7)
 
 NAME
 ----
-gitrevisions - specifying revisions and ranges for Git
+gitrevisions - Specifying revisions and ranges for Git
 
 SYNOPSIS
 --------
diff --git a/builtin/help.c b/builtin/help.c
index 0e0af8426a..5727fb5e51 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -402,38 +402,6 @@ static void show_html_page(const char *git_cmd)
 	open_html(page_path.buf);
 }
 
-static struct {
-	const char *name;
-	const char *help;
-} common_guides[] = {
-	{ "attributes", N_("Defining attributes per path") },
-	{ "everyday", N_("Everyday Git With 20 Commands Or So") },
-	{ "glossary", N_("A Git glossary") },
-	{ "ignore", N_("Specifies intentionally untracked files to ignore") },
-	{ "modules", N_("Defining submodule properties") },
-	{ "revisions", N_("Specifying revisions and ranges for Git") },
-	{ "tutorial", N_("A tutorial introduction to Git (for version 1.5.1 or newer)") },
-	{ "workflows", N_("An overview of recommended workflows with Git") },
-};
-
-static void list_common_guides_help(void)
-{
-	int i, longest = 0;
-
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		if (longest < strlen(common_guides[i].name))
-			longest = strlen(common_guides[i].name);
-	}
-
-	puts(_("The common Git guides are:\n"));
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		printf("   %s   ", common_guides[i].name);
-		mput_char(' ', longest - strlen(common_guides[i].name));
-		puts(_(common_guides[i].help));
-	}
-	putchar('\n');
-}
-
 static const char *check_git_cmd(const char* cmd)
 {
 	char *alias;
diff --git a/command-list.txt b/command-list.txt
index 2c81d8db74..1835f1a928 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -149,3 +149,11 @@ gitweb                                  ancillaryinterrogators
 git-whatchanged                         ancillaryinterrogators
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
+gitattributes                           guide
+giteveryday                             guide
+gitglossary                             guide
+gitignore                               guide
+gitmodules                              guide
+gitrevisions                            guide
+gittutorial                             guide
+gitworkflows                            guide
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 05722b1392..86d59419b3 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -58,7 +58,11 @@ command_list "$1" |
 sort |
 while read cmd category tags
 do
-	prefix=git-
+	if [ "$category" = guide ]; then
+		prefix=git
+	else
+		prefix=git-
+	fi
 	name=$(echo $cmd | sed "s/^${prefix}//")
 	sed -n '
 		/^NAME/,/'"$cmd"'/H
diff --git a/help.c b/help.c
index ae5281601a..88127fdd6f 100644
--- a/help.c
+++ b/help.c
@@ -314,6 +314,28 @@ static void list_commands_by_category(int cat, struct cmdname_help *cmds,
 	}
 }
 
+void list_common_guides_help(void)
+{
+	int i, longest = 0;
+	int nr = ARRAY_SIZE(command_list);
+	struct cmdname_help *cmds = command_list;
+
+	QSORT(cmds, nr, cmd_category_cmp);
+
+	for (i = 0; i < nr; i++) {
+		struct cmdname_help *cmd = cmds + i;
+
+		if (cmd->category != CAT_guide)
+			continue;
+		if (longest < strlen(cmd->name))
+			longest = strlen(cmd->name);
+	}
+
+	puts(_("The common Git guides are:\n"));
+	list_commands_by_category(CAT_guide, cmds, nr, longest);
+	putchar('\n');
+}
+
 void list_all_cmds_help(void)
 {
 	int i, longest = 0;
diff --git a/help.h b/help.h
index 62449f1b7e..de094e9e65 100644
--- a/help.h
+++ b/help.h
@@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds_help(void);
+extern void list_common_guides_help(void);
 extern void list_all_cmds(void);
 extern void list_porcelain_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
-- 
2.17.0.367.g5dd2e386c3
Eric Sunshine· Apr 23, 2018, 06:07 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v3 6/6] help: use command-list.txt for the source of guides

On Sat, Apr 21, 2018 at 12:54 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 13 quoted lines
> The help command currently hard codes the list of guides and their
> summary in C. Let's move this list to command-list.txt. This lets us
> extract summary lines from Documentation/git*.txt. This also
> potentially lets us lists guides in git.txt, but I'll leave that for
> now.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> @@ -58,7 +58,11 @@ command_list "$1" |
>  do
> -       prefix=git-
> +       if [ "$category" = guide ]; then
Style:
    if test "$category" = guide
    then
        ...
> +               prefix=git
> +       else
> +               prefix=git-
> +       fi
Nguyễn Thái Ngọc Duy· Apr 21, 2018, 16:54 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v3 5/6] help: add "-a --verbose" to list all commands with synopsis

This lists all recognized commands [1] by category. The group order follows closely git.txt.

[1] We may actually show commands that are not built (e.g. if you set NO_PERL you don't have git-instaweb but it's still listed here). I ignore the problem because on Linux a git package could be split anyway. The "git-core" package may not contain git-instaweb even if it's built because it may end up in a separate package. We can't know anyway.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-help.txt |  4 ++-
 builtin/help.c             |  7 ++++
 help.c                     | 69 ++++++++++++++++++++++++++++++++++++++
 help.h                     |  1 +
 t/t0012-help.sh            |  9 +++++
 5 files changed, 89 insertions(+), 1 deletion(-)
Show changes to 5 files +89 −1

Documentation/git-help.txt, builtin/help.c, help.c, help.h, t/t0012-help.sh

diff --git a/Documentation/git-help.txt b/Documentation/git-help.txt
index 40d328a4b3..a40fc38d8b 100644
--- a/Documentation/git-help.txt
+++ b/Documentation/git-help.txt
@@ -8,7 +8,7 @@ git-help - Display help information about Git
 SYNOPSIS
 --------
 [verse]
-'git help' [-a|--all] [-g|--guide]
+'git help' [-a|--all [--verbose]] [-g|--guide]
 	   [-i|--info|-m|--man|-w|--web] [COMMAND|GUIDE]
 
 DESCRIPTION
@@ -42,6 +42,8 @@ OPTIONS
 --all::
 	Prints all the available commands on the standard output. This
 	option overrides any given command or guide name.
+	When used with `--verbose` print description for all recognized
+	commands.
 
 -g::
 --guides::
diff --git a/builtin/help.c b/builtin/help.c
index 598867cfea..0e0af8426a 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -36,6 +36,7 @@ static const char *html_path;
 
 static int show_all = 0;
 static int show_guides = 0;
+static int verbose;
 static unsigned int colopts;
 static enum help_format help_format = HELP_FORMAT_NONE;
 static int exclude_guides;
@@ -48,6 +49,7 @@ static struct option builtin_help_options[] = {
 			HELP_FORMAT_WEB),
 	OPT_SET_INT('i', "info", &help_format, N_("show info page"),
 			HELP_FORMAT_INFO),
+	OPT__VERBOSE(&verbose, N_("print command description")),
 	OPT_END(),
 };
 
@@ -463,6 +465,11 @@ int cmd_help(int argc, const char **argv, const char *prefix)
 
 	if (show_all) {
 		git_config(git_help_config, NULL);
+		if (verbose) {
+			setup_pager();
+			list_all_cmds_help();
+			return 0;
+		}
 		printf(_("usage: %s%s"), _(git_usage_string), "\n\n");
 		load_command_list("git-", &main_cmds, &other_cmds);
 		list_commands(colopts, &main_cmds, &other_cmds);
diff --git a/help.c b/help.c
index 6688653f99..ae5281601a 100644
--- a/help.c
+++ b/help.c
@@ -285,6 +285,75 @@ void list_porcelain_cmds(void)
 	}
 }
 
+static int cmd_category_cmp(const void *elem1, const void *elem2)
+{
+	const struct cmdname_help *e1 = elem1;
+	const struct cmdname_help *e2 = elem2;
+
+	if (e1->category < e2->category)
+		return -1;
+	if (e1->category > e2->category)
+		return 1;
+	return strcmp(e1->name, e2->name);
+}
+
+static void list_commands_by_category(int cat, struct cmdname_help *cmds,
+				      int nr, int longest)
+{
+	int i;
+
+	for (i = 0; i < nr; i++) {
+		struct cmdname_help *cmd = cmds + i;
+
+		if (cmd->category != cat)
+			continue;
+
+		printf("   %s   ", cmd->name);
+		mput_char(' ', longest - strlen(cmd->name));
+		puts(_(cmd->help));
+	}
+}
+
+void list_all_cmds_help(void)
+{
+	int i, longest = 0;
+	int nr = ARRAY_SIZE(command_list);
+	struct cmdname_help *cmds = command_list;
+
+	for (i = 0; i < nr; i++) {
+		struct cmdname_help *cmd = cmds + i;
+
+		if (longest < strlen(cmd->name))
+			longest = strlen(cmd->name);
+	}
+
+	QSORT(cmds, nr, cmd_category_cmp);
+
+	printf("%s\n\n", _("Main Porcelain Commands"));
+	list_commands_by_category(CAT_mainporcelain, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Ancillary Commands / Manipulators"));
+	list_commands_by_category(CAT_ancillarymanipulators, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Ancillary Commands / Interrogators"));
+	list_commands_by_category(CAT_ancillaryinterrogators, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Interacting with Others"));
+	list_commands_by_category(CAT_foreignscminterface, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Low-level Commands / Manipulators"));
+	list_commands_by_category(CAT_plumbingmanipulators, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Low-level Commands / Interrogators"));
+	list_commands_by_category(CAT_plumbinginterrogators, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Low-level Commands / Synching Repositories"));
+	list_commands_by_category(CAT_synchingrepositories, cmds, nr, longest);
+
+	printf("\n%s\n\n", _("Low-level Commands / Internal Helpers"));
+	list_commands_by_category(CAT_purehelpers, cmds, nr, longest);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 33e2210ebd..62449f1b7e 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds_help(void);
 extern void list_all_cmds(void);
 extern void list_porcelain_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index fd2a7f27dc..53208ab20e 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -25,6 +25,15 @@ test_expect_success "setup" '
 	EOF
 '
 
+# make sure to exercise these code paths, the output is a bit tricky
+# to verify
+test_expect_success 'basic help commands' '
+	git help >/dev/null &&
+	git help -a >/dev/null &&
+	git help -g >/dev/null &&
+	git help -av >/dev/null
+'
+
 test_expect_success "works for commands and guides by default" '
 	configure_help &&
 	git help status &&
-- 
2.17.0.367.g5dd2e386c3
Nguyễn Thái Ngọc Duy· Apr 21, 2018, 16:54 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v3 4/6] git.c: implement --list-cmds=porcelain

This is useful for git-completion.bash because it needs this set of commands. Right now we have to maintain a separate command category in there.

Note that the current completion script incorrectly classifies filter-branch as porcelain and t9902 tests this behavior. We keep it this way in t9902 because this test does not really care which particular command is porcelain or plubmbing, they're just names.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash | 106 +++----------------------
 git.c                                  |   2 +
 help.c                                 |  12 +++
 help.h                                 |   1 +
 t/t9902-completion.sh                  |   6 +-
 5 files changed, 31 insertions(+), 96 deletions(-)
Show changes to 5 files +31 −96

contrib/completion/git-completion.bash, git.c, help.c, help.h, t/t9902-completion.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a5f13ade20..7d17ca23f6 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -835,18 +835,23 @@ __git_complete_strategy ()
 }
 
 __git_commands () {
-	if test -n "${GIT_TESTING_COMMAND_COMPLETION:-}"
+	if test -n "$GIT_TESTING_COMPLETION"
 	then
-		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
+		case "$1" in
+		porcelain)
+			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST";;
+		all)
+			printf "%s" "$GIT_TESTING_ALL_COMMAND_LIST";;
+		esac
 	else
-		git --list-cmds=all
+		git --list-cmds=$1
 	fi
 }
 
-__git_list_all_commands ()
+__git_list_commands ()
 {
 	local i IFS=" "$'\n'
-	for i in $(__git_commands)
+	for i in $(__git_commands $1)
 	do
 		case $i in
 		*--*)             : helper pattern;;
@@ -859,101 +864,14 @@ __git_all_commands=
 __git_compute_all_commands ()
 {
 	test -n "$__git_all_commands" ||
-	__git_all_commands=$(__git_list_all_commands)
-}
-
-__git_list_porcelain_commands ()
-{
-	local i IFS=" "$'\n'
-	__git_compute_all_commands
-	for i in $__git_all_commands
-	do
-		case $i in
-		*--*)             : helper pattern;;
-		applymbox)        : ask gittus;;
-		applypatch)       : ask gittus;;
-		archimport)       : import;;
-		cat-file)         : plumbing;;
-		check-attr)       : plumbing;;
-		check-ignore)     : plumbing;;
-		check-mailmap)    : plumbing;;
-		check-ref-format) : plumbing;;
-		checkout-index)   : plumbing;;
-		column)           : internal helper;;
-		commit-tree)      : plumbing;;
-		count-objects)    : infrequent;;
-		credential)       : credentials;;
-		credential-*)     : credentials helper;;
-		cvsexportcommit)  : export;;
-		cvsimport)        : import;;
-		cvsserver)        : daemon;;
-		daemon)           : daemon;;
-		diff-files)       : plumbing;;
-		diff-index)       : plumbing;;
-		diff-tree)        : plumbing;;
-		fast-import)      : import;;
-		fast-export)      : export;;
-		fsck-objects)     : plumbing;;
-		fetch-pack)       : plumbing;;
-		fmt-merge-msg)    : plumbing;;
-		for-each-ref)     : plumbing;;
-		hash-object)      : plumbing;;
-		http-*)           : transport;;
-		index-pack)       : plumbing;;
-		init-db)          : deprecated;;
-		local-fetch)      : plumbing;;
-		ls-files)         : plumbing;;
-		ls-remote)        : plumbing;;
-		ls-tree)          : plumbing;;
-		mailinfo)         : plumbing;;
-		mailsplit)        : plumbing;;
-		merge-*)          : plumbing;;
-		mktree)           : plumbing;;
-		mktag)            : plumbing;;
-		pack-objects)     : plumbing;;
-		pack-redundant)   : plumbing;;
-		pack-refs)        : plumbing;;
-		parse-remote)     : plumbing;;
-		patch-id)         : plumbing;;
-		prune)            : plumbing;;
-		prune-packed)     : plumbing;;
-		quiltimport)      : import;;
-		read-tree)        : plumbing;;
-		receive-pack)     : plumbing;;
-		remote-*)         : transport;;
-		rerere)           : plumbing;;
-		rev-list)         : plumbing;;
-		rev-parse)        : plumbing;;
-		runstatus)        : plumbing;;
-		sh-setup)         : internal;;
-		shell)            : daemon;;
-		show-ref)         : plumbing;;
-		send-pack)        : plumbing;;
-		show-index)       : plumbing;;
-		ssh-*)            : transport;;
-		stripspace)       : plumbing;;
-		symbolic-ref)     : plumbing;;
-		unpack-file)      : plumbing;;
-		unpack-objects)   : plumbing;;
-		update-index)     : plumbing;;
-		update-ref)       : plumbing;;
-		update-server-info) : daemon;;
-		upload-archive)   : plumbing;;
-		upload-pack)      : plumbing;;
-		write-tree)       : plumbing;;
-		var)              : infrequent;;
-		verify-pack)      : infrequent;;
-		verify-tag)       : plumbing;;
-		*) echo $i;;
-		esac
-	done
+	__git_all_commands=$(__git_list_commands all)
 }
 
 __git_porcelain_commands=
 __git_compute_porcelain_commands ()
 {
 	test -n "$__git_porcelain_commands" ||
-	__git_porcelain_commands=$(__git_list_porcelain_commands)
+	__git_porcelain_commands=$(__git_list_commands porcelain)
 }
 
 # Lists all set config variables starting with the given section prefix,
diff --git a/git.c b/git.c
index 64f67e7f7f..153f56fb95 100644
--- a/git.c
+++ b/git.c
@@ -230,6 +230,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 				list_builtins(NO_PARSEOPT, ' ');
 			else if (!strcmp(cmd, "all"))
 				list_all_cmds();
+			else if (!strcmp(cmd, "porcelain"))
+				list_porcelain_cmds();
 			else
 				die("unsupported command listing type '%s'", cmd);
 			exit(0);
diff --git a/help.c b/help.c
index e63006c333..6688653f99 100644
--- a/help.c
+++ b/help.c
@@ -273,6 +273,18 @@ void list_all_cmds(void)
 	clean_cmdnames(&other_cmds);
 }
 
+void list_porcelain_cmds(void)
+{
+	int i, nr = ARRAY_SIZE(command_list);
+	struct cmdname_help *cmds = command_list;
+
+	for (i = 0; i < nr; i++) {
+		if (cmds[i].category != CAT_mainporcelain)
+			continue;
+		puts(cmds[i].name);
+	}
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 0bf29f8dc5..33e2210ebd 100644
--- a/help.h
+++ b/help.h
@@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds(void);
+extern void list_porcelain_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 1b34caa1e1..5a23a46fcf 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -13,7 +13,7 @@ complete ()
 	return 0
 }
 
-# Be careful when updating this list:
+# Be careful when updating these lists:
 #
 # (1) The build tree may have build artifact from different branch, or
 #     the user's $PATH may have a random executable that may begin
@@ -30,7 +30,9 @@ complete ()
 #     completion for "git <TAB>", and a plumbing is excluded.  "add",
 #     "filter-branch" and "ls-files" are listed for this.
 
-GIT_TESTING_COMMAND_COMPLETION='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_COMPLETION=t
+GIT_TESTING_ALL_COMMAND_LIST='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_PORCELAIN_COMMAND_LIST='add checkout filter-branch'
 
 . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash"
 
-- 
2.17.0.367.g5dd2e386c3
SZEDER Gábor· Apr 23, 2018, 13:32 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v3 4/6] git.c: implement --list-cmds=porcelain

On Sat, Apr 21, 2018 at 6:54 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
> This is useful for git-completion.bash because it needs this set of
> commands. Right now we have to maintain a separate command category in
> there.

I don't really understand this paragraph, in particular its second sentence. I would have described this change like this:

  To get the list of commands offered after 'git <TAB>', the
  completion script filters out plumbing commands from the list of all
  git commands.  To do so, it contains a long hard-coded list of the
  names of all known plumbing commands, which is redundant with the
  categorization in 'command-list.txt', is a maintenance burden, and
  tends to get out-of-sync when new plumbing commands are added.
  Implement 'git --list-cmds=porcelain' to list only commands
  categorized as porcelains, so we can get rid of that long hard-coded
  command list from the completion script.

But then I noticed that it's not an accurate description of the current situation, because there is a wide grey area between porcelains and plumbing, and the completion script doesn't "filter out plumbing commands", but rather filters out commands that can be considered too low-level to be useful for "general" usage. Consequently, after 'git <TAB>' we also list:

  - some 'ancillaryinterrogators': blame, annotate, difftool, fsck,
    help
  - some 'ancillarymanipulators': config, mergetool, remote
  - some 'foreignscminterface': p4, request-pull, svn, send-email
  - even some plumbing: apply, name-rev (though 'name-rev' could be
    omitted; we have 'git describe')
  - and also all "unknown" 'git-foo' commands that can be found in
    $PATH, which can be the user's own git scripts or other
    git-related tools ('git-annex', Git LFS, etc.).

With this change we wouldn't list any of the above commands, but only those that are explicitly categorized as 'mainporcelain'. I'd much prefer the current behaviour.

Show 18 quoted lines
> Note that the current completion script incorrectly classifies
> filter-branch as porcelain and t9902 tests this behavior. We keep it
> this way in t9902 because this test does not really care which
> particular command is porcelain or plubmbing, they're just names.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  contrib/completion/git-completion.bash | 106 +++----------------------
>  git.c                                  |   2 +
>  help.c                                 |  12 +++
>  help.h                                 |   1 +
>  t/t9902-completion.sh                  |   6 +-
>  5 files changed, 31 insertions(+), 96 deletions(-)
>
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index a5f13ade20..7d17ca23f6 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
Show 16 quoted lines
> @@ -859,101 +864,14 @@ __git_all_commands=
>  __git_compute_all_commands ()
>  {
>         test -n "$__git_all_commands" ||
> -       __git_all_commands=$(__git_list_all_commands)
> -}
> -
> -__git_list_porcelain_commands ()
> -{
> -       local i IFS=" "$'\n'
> -       __git_compute_all_commands
> -       for i in $__git_all_commands
> -       do
> -               case $i in
> -               *--*)             : helper pattern;;
> -               applymbox)        : ask gittus;;
<...>
> -               verify-tag)       : plumbing;;
I will be glad to see this enormous list go.
Show 5 quoted lines
> -               *) echo $i;;
> -               esac
> -       done
> +       __git_all_commands=$(__git_list_commands all)
>  }
Duy Nguyen· Apr 24, 2018, 16:12 UTC · re: SZEDER Gábor · lore

Re: [PATCH v3 4/6] git.c: implement --list-cmds=porcelain

On Mon, Apr 23, 2018 at 3:32 PM, SZEDER Gábor <szeder.dev@gmail.com> wrote:
Show 20 quoted lines
> But then I noticed that it's not an accurate description of the
> current situation, because there is a wide grey area between
> porcelains and plumbing, and the completion script doesn't "filter out
> plumbing commands", but rather filters out commands that can be
> considered too low-level to be useful for "general" usage.
> Consequently, after 'git <TAB>' we also list:
>
>   - some 'ancillaryinterrogators': blame, annotate, difftool, fsck,
>     help
>   - some 'ancillarymanipulators': config, mergetool, remote
>   - some 'foreignscminterface': p4, request-pull, svn, send-email
>   - even some plumbing: apply, name-rev (though 'name-rev' could be
>     omitted; we have 'git describe')
>   - and also all "unknown" 'git-foo' commands that can be found in
>     $PATH, which can be the user's own git scripts or other
>     git-related tools ('git-annex', Git LFS, etc.).
>
> With this change we wouldn't list any of the above commands, but only
> those that are explicitly categorized as 'mainporcelain'.  I'd much
> prefer the current behaviour.

Yeah I noticed this (kinda) with filter-branch but I did not look further to see all this. It's good that you review this series then :)

For the first group (known commands), how about we add a new category "completion" in command-list.txt? Each command may belong to multiple categories (and my updated script has to deal with that [1]). For the second group, we could also have a special "external" category that is produced at run time, not specified in command-list.txt. --list-cmds option either has to accept multiple values, or we accept multiple --list-cmds=<category> options.

git-completion.bash will be updated to ask git "give me the commands in the mainporcelain, completable or external category". This also addresses another thing that bugs me: I wanted an option to let me complete all commands instead of just porcelain. This approach kinda generalizes that and it would be easy to let the user choose what category they want to complete.

[1] which also means I could bring "deprecated" category back.
-- 
Duy
Duy Nguyen· Apr 24, 2018, 16:17 UTC · re: Duy Nguyen · lore

Re: [PATCH v3 4/6] git.c: implement --list-cmds=porcelain

On Tue, Apr 24, 2018 at 6:12 PM, Duy Nguyen <pclouds@gmail.com> wrote:
Show 6 quoted lines
> git-completion.bash will be updated to ask git "give me the commands
> in the mainporcelain, completable or external category". This also
> addresses another thing that bugs me: I wanted an option to let me
> complete all commands instead of just porcelain. This approach kinda
> generalizes that and it would be easy to let the user choose what
> category they want to complete.

To complete this: there could also be yet another special category "custom", where --list-cmds=custom simply returns a list of commands specified in config file. With this the user can pick the "custom" category to have total control of what command shows up at "git <tab>" if they are not satisfied with the predefined categories.

-- 
Duy
SZEDER Gábor· Apr 25, 2018, 13:46 UTC · re: Duy Nguyen · lore

Re: [PATCH v3 4/6] git.c: implement --list-cmds=porcelain

On Tue, Apr 24, 2018 at 6:17 PM, Duy Nguyen <pclouds@gmail.com> wrote:
Show 13 quoted lines
> On Tue, Apr 24, 2018 at 6:12 PM, Duy Nguyen <pclouds@gmail.com> wrote:
>> git-completion.bash will be updated to ask git "give me the commands
>> in the mainporcelain, completable or external category". This also
>> addresses another thing that bugs me: I wanted an option to let me
>> complete all commands instead of just porcelain. This approach kinda
>> generalizes that and it would be easy to let the user choose what
>> category they want to complete.
>
> To complete this: there could also be yet another special category
> "custom", where --list-cmds=custom simply returns a list of commands
> specified in config file. With this the user can pick the "custom"
> category to have total control of what command shows up at "git <tab>"
> if they are not satisfied with the predefined categories.

Note that config settings can be repository-specific, which might cause surprises if the user sets a custom command list in one repository's config file, but not (or a different one!) in others. Then the list of completed commands will depend on where the user happened to be when first hitting 'git <TAB>'. Unless you forgo caching the list of commands and run 'git --list-cmds=...' every time the user hits 'git <TAB>', but that will add the overhead of fork()ing a subshell and a git command.

Once upon a time I toyed with the idea of introducing environment variables like $GIT_COMPLETION_EXCLUDE_COMMANDS and $GIT_COMPLETION_INCLUDE_COMMANDS, to exclude and include commands from 'git <TAB>'. I wanted to exclude 'git cherry', because I have never ever used it but it's always in the way when I want to cherry-pick. And I wanted to include 'git for-each-ref' back when I was running it frequently while working on speeding up refs completion, but it wouldn't have brought that much benefits, because I have a 'git for-each-commit' script, too. Never really pursued it, though, just patched the long list in __git_list_porcelain_commands() in my git clone :)

Duy Nguyen· Apr 25, 2018, 14:44 UTC · re: SZEDER Gábor · lore

Re: [PATCH v3 4/6] git.c: implement --list-cmds=porcelain

On Wed, Apr 25, 2018 at 3:46 PM, SZEDER Gábor <szeder.dev@gmail.com> wrote:
Show 20 quoted lines
> On Tue, Apr 24, 2018 at 6:17 PM, Duy Nguyen <pclouds@gmail.com> wrote:
>> On Tue, Apr 24, 2018 at 6:12 PM, Duy Nguyen <pclouds@gmail.com> wrote:
>>> git-completion.bash will be updated to ask git "give me the commands
>>> in the mainporcelain, completable or external category". This also
>>> addresses another thing that bugs me: I wanted an option to let me
>>> complete all commands instead of just porcelain. This approach kinda
>>> generalizes that and it would be easy to let the user choose what
>>> category they want to complete.
>>
>> To complete this: there could also be yet another special category
>> "custom", where --list-cmds=custom simply returns a list of commands
>> specified in config file. With this the user can pick the "custom"
>> category to have total control of what command shows up at "git <tab>"
>> if they are not satisfied with the predefined categories.
>
> Note that config settings can be repository-specific, which might
> cause surprises if the user sets a custom command list in one
> repository's config file, but not (or a different one!) in others.
> Then the list of completed commands will depend on where the user
> happened to be when first hitting 'git <TAB>'.

I think that is expected when the word "config file" is mentioned. It's no different than aliases, which could also be repo specific and affects completion.

> Unless you forgo
> caching the list of commands and run 'git --list-cmds=...' every time
> the user hits 'git <TAB>', but that will add the overhead of fork()ing
> a subshell and a git command.

This is a good point. I'd rather forgo caching iff the "custom" strategy is used (technically we could still cache if we know the source if $HOME/.gitconfig or higher but it's not worth the effort). Just make it clear to the user that if they go with "custom" strategy then they may hit some performance hit.

Show 11 quoted lines
> Once upon a time I toyed with the idea of introducing environment
> variables like $GIT_COMPLETION_EXCLUDE_COMMANDS and
> $GIT_COMPLETION_INCLUDE_COMMANDS, to exclude and include commands from
> 'git <TAB>'.  I wanted to exclude 'git cherry', because I have never
> ever used it but it's always in the way when I want to cherry-pick.
> And I wanted to include 'git for-each-ref' back when I was running it
> frequently while working on speeding up refs completion, but it
> wouldn't have brought that much benefits, because I have a 'git
> for-each-commit' script, too.
> Never really pursued it, though, just patched the long list in
> __git_list_porcelain_commands() in my git clone :)

I'm tempted to support "delta custom list" (e.g. "+cherry-pick" in the config variable means "add that command on the existing list", or "-blame" means exclude it) but that's probably too much work.

-- 
Duy
SZEDER Gábor· Apr 25, 2018, 13:06 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v3 4/6] git.c: implement --list-cmds=porcelain

On Sat, Apr 21, 2018 at 6:54 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
Show 5 quoted lines
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index a5f13ade20..7d17ca23f6 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -835,18 +835,23 @@ __git_complete_strategy ()
> +__git_list_commands ()

Please add a comment before this function to mention its mandatory argument and that is should correspond to a category in 'command-list.txt'

> -__git_list_all_commands ()
> -__git_list_porcelain_commands ()

Users can have their own completion scriptlets for their own git commands, and these should be able to rely on helper functions in git-completion.bash to do things like refs completion and what not. Therefore, we tend not to remove or alter those helper functions in a backwards incompatible way, because we don't want to break those completion scriptlets.

Your patch removes these two functions. At first I let it slide, because first I thought that anyone who needs the list of all git commands is better off calling __git_compute_all_commands() and then using $__git_all_commands (same goes for porcelains), because it involves less fork()ed subshells and external processes. Then I thought why would anyone possibly need the list of git commands in a custom completion scriptlet in the first place... what for?!

Well, it turns out that there is at least one completion script out there that does rely on __git_list_all_commands() [1].

Please keep these two functions as a thin wrapper around __git_list_commands().

[1] And in a rather curious way at that:
    https://github.com/github/hub/blob/master/etc/hub.bash_completion.sh#L12
Duy Nguyen· Apr 25, 2018, 14:39 UTC · re: SZEDER Gábor · lore

Re: [PATCH v3 4/6] git.c: implement --list-cmds=porcelain

On Wed, Apr 25, 2018 at 3:06 PM, SZEDER Gábor <szeder.dev@gmail.com> wrote:
Show 10 quoted lines
>> -__git_list_all_commands ()
>
>> -__git_list_porcelain_commands ()
>
> Users can have their own completion scriptlets for their own git
> commands, and these should be able to rely on helper functions in
> git-completion.bash to do things like refs completion and what not.
> Therefore, we tend not to remove or alter those helper functions in a
> backwards incompatible way, because we don't want to break those
> completion scriptlets.

What kind of "API" guarantee do we provide here? I don't mind adding the wrappers, but for me it's hard for new contributors (like me) to see which one should be stable and which one is internal.

-- 
Duy
Duy Nguyen· Apr 21, 2018, 16:56 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v3 0/6] Keep all info in command-list.txt in git binary

On Sat, Apr 21, 2018 at 06:54:08PM +0200, Nguyễn Thái Ngọc Duy wrote:
Show 7 quoted lines
> Changes:
> 
> - remove the deprecated column in command-list.txt. My change break it
>   anyway if anyone uses it.
> - fix up failed tests that I marked in the RFC and kinda forgot about it.
> - fix bashisms in generate-cmdlist.sh
> - fix segfaul in "git help"
Sorry I forgot the interdiff
Show changes to 6 files +34 −24

command-list.txt, contrib/completion/git-completion.bash, generate-cmdlist.sh, help.c, t/t0012-help.sh, t/t9902-completion.sh

diff --git a/command-list.txt b/command-list.txt
index 0809a19184..1835f1a928 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -9,7 +9,7 @@ history      grow, mark and tweak your common history
 remote       collaborate (see also: git help workflows)
 
 ### command list (do not change this line)
-# command name                          category [deprecated] [common]
+# command name                          category                [common]
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 9f17703aa7..7d17ca23f6 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -835,19 +835,23 @@ __git_complete_strategy ()
 }
 
 __git_commands () {
-	if test -n "${GIT_TESTING_COMMAND_COMPLETION:-}"
+	if test -n "$GIT_TESTING_COMPLETION"
 	then
-		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
+		case "$1" in
+		porcelain)
+			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST";;
+		all)
+			printf "%s" "$GIT_TESTING_ALL_COMMAND_LIST";;
+		esac
 	else
 		git --list-cmds=$1
 	fi
 }
 
-__git_list_all_commands ()
+__git_list_commands ()
 {
 	local i IFS=" "$'\n'
-	local category=${1-all}
-	for i in $(__git_commands $category)
+	for i in $(__git_commands $1)
 	do
 		case $i in
 		*--*)             : helper pattern;;
@@ -860,14 +864,14 @@ __git_all_commands=
 __git_compute_all_commands ()
 {
 	test -n "$__git_all_commands" ||
-	__git_all_commands=$(__git_list_all_commands)
+	__git_all_commands=$(__git_list_commands all)
 }
 
 __git_porcelain_commands=
 __git_compute_porcelain_commands ()
 {
 	test -n "$__git_porcelain_commands" ||
-	__git_porcelain_commands=$(__git_list_all_commands porcelain)
+	__git_porcelain_commands=$(__git_list_commands porcelain)
 }
 
 # Lists all set config variables starting with the given section prefix,
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index e35f3e357b..86d59419b3 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -36,7 +36,7 @@ sed -n '
 	' "$1"
 printf '};\n\n'
 
-echo "#define GROUP_NONE 0xff /* no common group */"
+echo "#define GROUP_NONE 0xff"
 n=0
 while read grp
 do
@@ -45,15 +45,6 @@ do
 done <"$grps"
 echo
 
-echo '/*'
-printf 'static const char *cmd_categories[] = {\n'
-category_list "$1" |
-while read category; do
-	printf '\t\"'$category'\",\n'
-done
-printf '\tNULL\n};\n\n'
-echo '*/'
-
 n=0
 category_list "$1" |
 while read category; do
@@ -68,10 +59,11 @@ sort |
 while read cmd category tags
 do
 	if [ "$category" = guide ]; then
-		name=${cmd/git}
+		prefix=git
 	else
-		name=${cmd/git-}
+		prefix=git-
 	fi
+	name=$(echo $cmd | sed "s/^${prefix}//")
 	sed -n '
 		/^NAME/,/'"$cmd"'/H
 		${
diff --git a/help.c b/help.c
index a44f4a113e..88127fdd6f 100644
--- a/help.c
+++ b/help.c
@@ -201,7 +201,8 @@ static void extract_common_cmds(struct cmdname_help **p_common_cmds,
 	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
 		const struct cmdname_help *cmd = command_list + i;
 
-		if (cmd->category != CAT_mainporcelain)
+		if (cmd->category != CAT_mainporcelain ||
+		    cmd->group == GROUP_NONE)
 			continue;
 
 		common_cmds[nr++] = *cmd;
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index fd2a7f27dc..53208ab20e 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -25,6 +25,15 @@ test_expect_success "setup" '
 	EOF
 '
 
+# make sure to exercise these code paths, the output is a bit tricky
+# to verify
+test_expect_success 'basic help commands' '
+	git help >/dev/null &&
+	git help -a >/dev/null &&
+	git help -g >/dev/null &&
+	git help -av >/dev/null
+'
+
 test_expect_success "works for commands and guides by default" '
 	configure_help &&
 	git help status &&
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 4bfd26ddf9..5a23a46fcf 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -13,7 +13,7 @@ complete ()
 	return 0
 }
 
-# Be careful when updating this list:
+# Be careful when updating these lists:
 #
 # (1) The build tree may have build artifact from different branch, or
 #     the user's $PATH may have a random executable that may begin
@@ -30,7 +30,9 @@ complete ()
 #     completion for "git <TAB>", and a plumbing is excluded.  "add",
 #     "filter-branch" and "ls-files" are listed for this.
 
-GIT_TESTING_COMMAND_COMPLETION='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_COMPLETION=t
+GIT_TESTING_ALL_COMMAND_LIST='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_PORCELAIN_COMMAND_LIST='add checkout filter-branch'
 
 . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash"
 
@@ -1208,6 +1210,8 @@ test_expect_success 'basic' '
 	grep -q "^add \$" out &&
 	# script
 	grep -q "^filter-branch \$" out &&
+	# plumbing
+	! grep -q "^ls-files \$" out &&
 
 	run_completion "git f" &&
 	! grep -q -v "^f" out
@@ -1270,7 +1274,7 @@ test_expect_success 'general options' '
 	test_completion "git --no-r" "--no-replace-objects "
 '
 
-test_expect_failure 'general options plus command' '
+test_expect_success 'general options plus command' '
 	test_completion "git --version check" "checkout " &&
 	test_completion "git --paginate check" "checkout " &&
 	test_completion "git --git-dir=foo check" "checkout " &&
Ramsay Jones· Apr 22, 2018, 14:45 UTC · re: Duy Nguyen · lore

Re: [PATCH v3 0/6] Keep all info in command-list.txt in git binary

On 21/04/18 17:56, Duy Nguyen wrote:
Show 11 quoted lines
> On Sat, Apr 21, 2018 at 06:54:08PM +0200, Nguyễn Thái Ngọc Duy wrote:
>> Changes:
>>
>> - remove the deprecated column in command-list.txt. My change break it
>>   anyway if anyone uses it.
>> - fix up failed tests that I marked in the RFC and kinda forgot about it.
>> - fix bashisms in generate-cmdlist.sh
>> - fix segfaul in "git help"
> 
> Sorry I forgot the interdiff
> 
[snip]
Show 17 quoted lines
> diff --git a/t/t0012-help.sh b/t/t0012-help.sh
> index fd2a7f27dc..53208ab20e 100755
> --- a/t/t0012-help.sh
> +++ b/t/t0012-help.sh
> @@ -25,6 +25,15 @@ test_expect_success "setup" '
>  	EOF
>  '
>  
> +# make sure to exercise these code paths, the output is a bit tricky
> +# to verify
> +test_expect_success 'basic help commands' '
> +	git help >/dev/null &&
> +	git help -a >/dev/null &&
> +	git help -g >/dev/null &&
> +	git help -av >/dev/null
> +'
> +
I think you need to try a little harder than this! ;-)

This test would not have noticed the recent failure (what annoyed me was that git build without error and then the test-suite sailed through with nary a squeak of complaint). Viz:

  $ ./git help
  usage: git [--version] [--help] [-C <path>] [-c <name>=<value>]
             [--exec-path[=<path>]] [--html-path] [--man-path] [--info-path]
             [-p | --paginate | --no-pager] [--no-replace-objects] [--bare]
             [--git-dir=<path>] [--work-tree=<path>] [--namespace=<name>]
             <command> [<args>]
  
  These are common Git commands used in various situations:
  
  'git help -a' and 'git help -g' list available subcommands and some
  concept guides. See 'git help <command>' or 'git help <concept>'
  to read about a specific subcommand or concept.
  $ echo $?
  0
  $ 
  $ ./git help -g
  The common Git guides are:
  
  
  'git help -a' and 'git help -g' list available subcommands and some
  concept guides. See 'git help <command>' or 'git help <concept>'
  to read about a specific subcommand or concept.
  $ echo $?
  0
  $ 
  $ ./git help -av
  Main Porcelain Commands
  
  
  Ancillary Commands / Manipulators
  
  
  Ancillary Commands / Interrogators
  
  
  Interacting with Others
  
  
  Low-level Commands / Manipulators
  
  
  Low-level Commands / Interrogators
  
  
  Low-level Commands / Synching Repositories
  
  
  Low-level Commands / Internal Helpers
  
  $ echo $?
  0
  $ 
 
I started to add some tests, like so:
  diff --git a/t/t0012-help.sh b/t/t0012-help.sh
  index fd2a7f27d..7e10c2862 100755
  --- a/t/t0012-help.sh
  +++ b/t/t0012-help.sh
  @@ -49,6 +49,22 @@ test_expect_success "--help does not work for guides" "
          test_i18ncmp expect actual
   "
   
  +test_expect_success 'git help' '
  +       git help >help.output &&
  +       test_i18ngrep "^   clone  " help.output &&
  +       test_i18ngrep "^   add    " help.output &&
  +       test_i18ngrep "^   log    " help.output &&
  +       test_i18ngrep "^   commit " help.output &&
  +       test_i18ngrep "^   fetch  " help.output
  +'
  +
  +test_expect_success 'git help -g' '
  +       git help -g >help.output &&
  +       test_i18ngrep "^   attributes " help.output &&
  +       test_i18ngrep "^   everyday   " help.output &&
  +       test_i18ngrep "^   tutorial   " help.output
  +'
  +
   test_expect_success 'generate builtin list' '
          git --list-cmds=builtins >builtins
   '

These tests, while not rigorous, did at least correctly catch the bad build for me. I was about to add the obvious one for 'git help -av', when I decided to catch the problem 'at source', viz:

  diff --git a/help.c b/help.c
  index a47f7e2c4..13790bb54 100644
  --- a/help.c
  +++ b/help.c
  @@ -196,6 +196,9 @@ static void extract_common_cmds(struct cmdname_help **p_common_cmds,
   	int i, nr = 0;
   	struct cmdname_help *common_cmds;
   
  +	if (ARRAY_SIZE(command_list) == 0)
  +		BUG("empty command_list");
  +
   	ALLOC_ARRAY(common_cmds, ARRAY_SIZE(command_list));
   
   	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
  @@ -279,6 +282,9 @@ void list_porcelain_cmds(void)
   	int i, nr = ARRAY_SIZE(command_list);
   	struct cmdname_help *cmds = command_list;
   
  +	if (nr == 0)
  +		BUG("empty command_list");
  +
   	for (i = 0; i < nr; i++) {
   		if (cmds[i].category != CAT_mainporcelain)
   			continue;
  @@ -321,6 +327,9 @@ void list_common_guides_help(void)
   	int nr = ARRAY_SIZE(command_list);
   	struct cmdname_help *cmds = command_list;
   
  +	if (nr == 0)
  +		BUG("empty command_list");
  +
   	QSORT(cmds, nr, cmd_category_cmp);
   
   	for (i = 0; i < nr; i++) {
  @@ -343,6 +352,9 @@ void list_all_cmds_help(void)
   	int nr = ARRAY_SIZE(command_list);
   	struct cmdname_help *cmds = command_list;
   
  +	if (nr == 0)
  +		BUG("empty command_list");
  +
   	for (i = 0; i < nr; i++) {
   		struct cmdname_help *cmd = cmds + i;
   
  
This had a very dramatic effect on the test-suite, since every single
test file failed while sourcing 'test-lib.sh'. [The test for having
built git ('"$GIT_BUILD_DIR/git" >/dev/null') tries to output help,
because you haven't given a command, and hits BUG - core dump!]

I haven't tried this patch series yet (I will hopefully find some time tonight), but it looks like it will fix the problem.

ATB, Ramsay Jones

Duy Nguyen· Apr 22, 2018, 15:22 UTC · re: Ramsay Jones · lore

Re: [PATCH v3 0/6] Keep all info in command-list.txt in git binary

On Sun, Apr 22, 2018 at 4:45 PM, Ramsay Jones <ramsay@ramsayjones.plus.com> wrote:

Show 34 quoted lines
>
>
> On 21/04/18 17:56, Duy Nguyen wrote:
>> On Sat, Apr 21, 2018 at 06:54:08PM +0200, Nguyễn Thái Ngọc Duy wrote:
>>> Changes:
>>>
>>> - remove the deprecated column in command-list.txt. My change break it
>>>   anyway if anyone uses it.
>>> - fix up failed tests that I marked in the RFC and kinda forgot about it.
>>> - fix bashisms in generate-cmdlist.sh
>>> - fix segfaul in "git help"
>>
>> Sorry I forgot the interdiff
>>
> [snip]
>
>> diff --git a/t/t0012-help.sh b/t/t0012-help.sh
>> index fd2a7f27dc..53208ab20e 100755
>> --- a/t/t0012-help.sh
>> +++ b/t/t0012-help.sh
>> @@ -25,6 +25,15 @@ test_expect_success "setup" '
>>       EOF
>>  '
>>
>> +# make sure to exercise these code paths, the output is a bit tricky
>> +# to verify
>> +test_expect_success 'basic help commands' '
>> +     git help >/dev/null &&
>> +     git help -a >/dev/null &&
>> +     git help -g >/dev/null &&
>> +     git help -av >/dev/null
>> +'
>> +
> I think you need to try a little harder than this! ;-)

Yeah. I did think about grepping the output but decided not to because of gettext poison stuff and column output in "git help". If we do want to test this, how about I extend --list-cmds= option to take a few more parameters? --list-cmds=common would output all common commands, --list-cmds=<category> does the same for other command category. This way we can verify without worrying about text formatting, paging or translation.

-- 
Duy
Ramsay Jones· Apr 22, 2018, 15:58 UTC · re: Duy Nguyen · lore

Re: [PATCH v3 0/6] Keep all info in command-list.txt in git binary

On 22/04/18 16:22, Duy Nguyen wrote:
Show 44 quoted lines
> On Sun, Apr 22, 2018 at 4:45 PM, Ramsay Jones
> <ramsay@ramsayjones.plus.com> wrote:
>>
>>
>> On 21/04/18 17:56, Duy Nguyen wrote:
>>> On Sat, Apr 21, 2018 at 06:54:08PM +0200, Nguyễn Thái Ngọc Duy wrote:
>>>> Changes:
>>>>
>>>> - remove the deprecated column in command-list.txt. My change break it
>>>>   anyway if anyone uses it.
>>>> - fix up failed tests that I marked in the RFC and kinda forgot about it.
>>>> - fix bashisms in generate-cmdlist.sh
>>>> - fix segfaul in "git help"
>>>
>>> Sorry I forgot the interdiff
>>>
>> [snip]
>>
>>> diff --git a/t/t0012-help.sh b/t/t0012-help.sh
>>> index fd2a7f27dc..53208ab20e 100755
>>> --- a/t/t0012-help.sh
>>> +++ b/t/t0012-help.sh
>>> @@ -25,6 +25,15 @@ test_expect_success "setup" '
>>>       EOF
>>>  '
>>>
>>> +# make sure to exercise these code paths, the output is a bit tricky
>>> +# to verify
>>> +test_expect_success 'basic help commands' '
>>> +     git help >/dev/null &&
>>> +     git help -a >/dev/null &&
>>> +     git help -g >/dev/null &&
>>> +     git help -av >/dev/null
>>> +'
>>> +
>> I think you need to try a little harder than this! ;-)
> 
> Yeah. I did think about grepping the output but decided not to because
> of gettext poison stuff and column output in "git help". If we do want
> to test this, how about I extend --list-cmds= option to take a few
> more parameters? --list-cmds=common would output all common commands,
> --list-cmds=<category> does the same for other command category. This
> way we can verify without worrying about text formatting, paging or
> translation.

Hmm, my immediate reaction would be to prefer my simple tests. Yes, they are not exactly rigorous and they would be affected by changing the help formatting, but they are effective. ;-)

[I don't think the formatting would change that often, or at all - whoever submits that patch would get to update the tests!]

What did you think about adding the BUG() checks?

ATB, Ramsay Jones

Duy Nguyen· Apr 22, 2018, 16:12 UTC · re: Ramsay Jones · lore

Re: [PATCH v3 0/6] Keep all info in command-list.txt in git binary

On Sun, Apr 22, 2018 at 5:58 PM, Ramsay Jones <ramsay@ramsayjones.plus.com> wrote:

Show 16 quoted lines
>>> I think you need to try a little harder than this! ;-)
>>
>> Yeah. I did think about grepping the output but decided not to because
>> of gettext poison stuff and column output in "git help". If we do want
>> to test this, how about I extend --list-cmds= option to take a few
>> more parameters? --list-cmds=common would output all common commands,
>> --list-cmds=<category> does the same for other command category. This
>> way we can verify without worrying about text formatting, paging or
>> translation.
>
> Hmm, my immediate reaction would be to prefer my simple tests.
> Yes, they are not exactly rigorous and they would be affected
> by changing the help formatting, but they are effective. ;-)
>
> [I don't think the formatting would change that often, or at
> all - whoever submits that patch would get to update the tests!]

Hmm.. for non-column output that's true. "git help" with column output should probably fine as well because even though we add more and more commands every month, these are not marked common (and unlikely so). So yeah I agree.

> What did you think about adding the BUG() checks?

I was thinking if there was a way to fail the build after running ./generate-cmds.sh and generating empty output but it does not sound easy to do. So yeah, BUG() checks sound good.

-- 
Duy
Ramsay Jones· Apr 22, 2018, 16:36 UTC · re: Duy Nguyen · lore

Re: [PATCH v3 0/6] Keep all info in command-list.txt in git binary

On 22/04/18 17:12, Duy Nguyen wrote:
Show 29 quoted lines
> On Sun, Apr 22, 2018 at 5:58 PM, Ramsay Jones
> <ramsay@ramsayjones.plus.com> wrote:
>>>> I think you need to try a little harder than this! ;-)
>>>
>>> Yeah. I did think about grepping the output but decided not to because
>>> of gettext poison stuff and column output in "git help". If we do want
>>> to test this, how about I extend --list-cmds= option to take a few
>>> more parameters? --list-cmds=common would output all common commands,
>>> --list-cmds=<category> does the same for other command category. This
>>> way we can verify without worrying about text formatting, paging or
>>> translation.
>>
>> Hmm, my immediate reaction would be to prefer my simple tests.
>> Yes, they are not exactly rigorous and they would be affected
>> by changing the help formatting, but they are effective. ;-)
>>
>> [I don't think the formatting would change that often, or at
>> all - whoever submits that patch would get to update the tests!]
> 
> Hmm.. for non-column output that's true. "git help" with column output
> should probably fine as well because even though we add more and more
> commands every month, these are not marked common (and unlikely so).
> So yeah I agree.
> 
>> What did you think about adding the BUG() checks?
> 
> I was thinking if there was a way to fail the build after running
> ./generate-cmds.sh and generating empty output but it does not sound
> easy to do. So yeah, BUG() checks sound good.

Yeah, failing the build would be preferable, but I didn't get that far. ;-)

[BTW, I just applied your patch series to the 'next' branch (I just couldn't be bothered to revert your last series from the 'pu' branch, etc.) and, as expected, everything built OK, passed the test-suite and 'git help' is working just fine.]

Thanks!

ATB, Ramsay Jones

Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:30 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 00/12] Keep all info in command-list.txt in git binary

This is not exactly v4 and likely broken. But I've made several debatable changes and would like your opinions before making even more changes in this direction.

In 03/12 I made a format change in command-list.txt. The group description is no longer in this file but in help.c instead. This simplifies the format. However it may be harder for people to know what category means what (but then it's already so for a lot more categories).

In 11/12 I added the new "complete" category to command-list.txt so that we could replace the giant list in git-completion.bash. This is really questionable because several commands will NOT be completable anymore. These are listed there so we can discuss.

Another thing I wonder is, whether I should tag "nocomplete" instead of "complete" in command-list.txt, similar to parseopt's nocomplete strategy: commands are completable by default then we just exclude some of them. The "complete" tag goes the opposite direction, we only show ones that are either external, "complete" or mainporcelain.

I think we would go with whitelist than blacklist though to avoid helper commands showing up by default...

Also in 02/12 I moved to fixed column format. Strictly speaking I do not need that (but it makes the code slightly more complex). If you are really against fixed column format, speak up now.

Nguyễn Thái Ngọc Duy (12):
  generate-cmds.sh: factor out synopsis extract code
  generate-cmds.sh: export all commands to command-list.h
  help: use command-list.h for common command list
  Remove common-cmds.h
  git.c: convert --list-*builtins to --list-cmds=*
  git: accept multiple --list-cmds options
  completion: implement and use --list-cmds=all
  git: support --list-cmds=<category>
  help: add "-a --verbose" to list all commands with synopsis
  help: use command-list.txt for the source of guides
  command-list.txt: add new category "complete"
  completion: let git provide the completable command list
 .gitignore                             |   2 +-
 Documentation/git-help.txt             |   4 +-
 Documentation/gitattributes.txt        |   2 +-
 Documentation/gitmodules.txt           |   2 +-
 Documentation/gitrevisions.txt         |   2 +-
 Makefile                               |  16 +--
 builtin/help.c                         |  39 +-----
 command-list.txt                       |  55 ++++----
 contrib/completion/git-completion.bash | 121 +++-------------
 generate-cmdlist.sh                    | 128 +++++++++++------
 git.c                                  |  19 ++-
 help.c                                 | 186 +++++++++++++++++++++----
 help.h                                 |   4 +
 t/t0012-help.sh                        |  11 +-
 t/t9902-completion.sh                  |   5 +-
 15 files changed, 344 insertions(+), 252 deletions(-)
-- 
2.17.0.519.gb89679a4aa
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:30 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 01/12] generate-cmds.sh: factor out synopsis extract code

This makes it easier to reuse the same code in another place (very soon).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 generate-cmdlist.sh | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
Show changes to generate-cmdlist.sh +13 −7
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index eeea4b67ea..17d6809ef5 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,17 @@
 #!/bin/sh
 
+get_synopsis () {
+	local cmd="$1"
+
+	sed -n '
+		/^NAME/,/'"$cmd"'/H
+		${
+			x
+			s/.*'"$cmd"' - \(.*\)/N_("\1")/
+			p
+		}' "Documentation/$cmd.txt"
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
 	char name[16];
@@ -39,12 +51,6 @@ sort |
 while read cmd tags
 do
 	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
-	sed -n '
-		/^NAME/,/git-'"$cmd"'/H
-		${
-			x
-			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
-			p
-		}' "Documentation/git-$cmd.txt"
+	echo "	{\"$cmd\", $(get_synopsis git-$cmd), $tag},"
 done
 echo "};"
-- 
2.17.0.519.gb89679a4aa
Eric Sunshine· Apr 25, 2018, 17:59 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v4/wip 01/12] generate-cmds.sh: factor out synopsis extract code

On Wed, Apr 25, 2018 at 12:30 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 9 quoted lines
> This makes it easier to reuse the same code in another place (very
> soon).
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> @@ -1,5 +1,17 @@
> +get_synopsis () {
> +       local cmd="$1"
'local' is a Bash-ism, isn't it?
Show 26 quoted lines
> +       sed -n '
> +               /^NAME/,/'"$cmd"'/H
> +               ${
> +                       x
> +                       s/.*'"$cmd"' - \(.*\)/N_("\1")/
> +                       p
> +               }' "Documentation/$cmd.txt"
> +}
> +
>  echo "/* Automatically generated by generate-cmdlist.sh */
>  struct cmdname_help {
>         char name[16];
> @@ -39,12 +51,6 @@ sort |
>  while read cmd tags
>  do
>         tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
> -       sed -n '
> -               /^NAME/,/git-'"$cmd"'/H
> -               ${
> -                       x
> -                       s/.*git-'"$cmd"' - \(.*\)/      {"'"$cmd"'", N_("\1"), '$tag'},/
> -                       p
> -               }' "Documentation/git-$cmd.txt"
> +       echo "  {\"$cmd\", $(get_synopsis git-$cmd), $tag},"
>  done
>  echo "};"
SZEDER Gábor· Apr 25, 2018, 18:13 UTC · re: Eric Sunshine · lore

Re: [PATCH v4/wip 01/12] generate-cmds.sh: factor out synopsis extract code

On Wed, Apr 25, 2018 at 7:59 PM Eric Sunshine <sunshine@sunshineco.com> wrote:

Show 11 quoted lines
> On Wed, Apr 25, 2018 at 12:30 PM, Nguyễn Thái Ngọc Duy
> <pclouds@gmail.com> wrote:
> > This makes it easier to reuse the same code in another place (very
> > soon).
> >
> > Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> > ---
> > diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> > @@ -1,5 +1,17 @@
> > +get_synopsis () {
> > +       local cmd="$1"
> 'local' is a Bash-ism, isn't it?

Well, strictly speaking it isn't, because many shells support it besides Bash.

I don't remember seeing any complaints about 01d3a526ad (t0000: check whether the shell supports the "local" keyword, 2017-10-26), but we only have that commit for a couple or months / two releases, so it's still too early to start using 'local' in build scripts.

Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:30 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 02/12] generate-cmds.sh: export all commands to command-list.h

The current generate-cmds.sh generates just enough to print "git help" output. That is, it only extracts help text for common commands.

The script is now updated to extract help text for all commands and keep command classification a new file, command-list.h. This will be useful later:

- "git help -a" could print a short summary of all commands instead of
  just the common ones.
- "git" could produce a list of commands of one or more category. One
  of its use is to reduce another command classification embedded in
  git-completion.bash.

The new file can be generated but is not used anywhere yet. The plan is we migrate away from common-cmds.h. Then we can kill off common-cmds.h build rules and generation code (and also delete duplicate content in command-list.h which we keep for now to not mess generate-cmds.sh up too much).

PS. The new fixed column requirement on command-list.txt is technically not needed. But it helps simplify the code a bit at this stage. We could lift this restriction later if we want to.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 .gitignore          |  1 +
 Makefile            | 13 ++++++---
 command-list.txt    |  4 +--
 generate-cmdlist.sh | 67 ++++++++++++++++++++++++++++++++++++++++++---
 4 files changed, 75 insertions(+), 10 deletions(-)
Show changes to 4 files +75 −10

.gitignore, Makefile, command-list.txt, generate-cmdlist.sh

diff --git a/.gitignore b/.gitignore
index 833ef3b0b7..d4c3914167 100644
--- a/.gitignore
+++ b/.gitignore
@@ -180,6 +180,7 @@
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
 /common-cmds.h
+/command-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index f181687250..2a8913ea21 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h
+GENERATED_H += common-cmds.h command-list.h
 
 LIB_H = $(shell $(FIND) . \
 	-name .git -prune -o \
@@ -1938,6 +1938,11 @@ $(BUILT_INS): git$X
 common-cmds.h: generate-cmdlist.sh command-list.txt
 
 common-cmds.h: $(wildcard Documentation/git-*.txt)
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON >$@+ && mv $@+ $@
+
+command-list.h: generate-cmdlist.sh command-list.txt
+
+command-list.h: $(wildcard Documentation/git-*.txt)
 	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
@@ -2148,7 +2153,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h
+# Dependencies on automatically generated headers such as common-cmds.h or command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2527,7 +2532,7 @@ sparse: $(SP_OBJ)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h
+check: common-cmds.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -2775,7 +2780,7 @@ clean: profile-clean coverage-clean
 	$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/command-list.txt b/command-list.txt
index a1fad28fd8..786536aba0 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -8,8 +8,8 @@ info         examine the history and state (see also: git help revisions)
 history      grow, mark and tweak your common history
 remote       collaborate (see also: git help workflows)
 
-### command list (do not change this line)
-# command name                          category [deprecated] [common]
+### command list (do not change this line, also do not change alignment)
+# command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 17d6809ef5..9ba9911f09 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,27 @@
 #!/bin/sh
 
+die () {
+	echo "$@" >&2
+	exit 1
+}
+
+command_list () {
+	sed '1,/^### command list/d;/^#/d' "$1"
+}
+
+get_categories() {
+	tr ' ' '\n'|
+	grep -v '^$' |
+	sort |
+	uniq
+}
+
+category_list () {
+	command_list "$1" |
+	cut -c 40- |
+	get_categories
+}
+
 get_synopsis () {
 	local cmd="$1"
 
@@ -12,14 +34,51 @@ get_synopsis () {
 		}' "Documentation/$cmd.txt"
 }
 
+define_categories() {
+	echo
+	echo "/* Command categories */"
+	bit=0
+	category_list "$1" |
+	while read cat
+	do
+		echo "#define CAT_$cat (1UL << $bit)"
+		bit=$(($bit+1))
+	done
+	test "$bit" -gt 32 && die "Urgh.. too many categories?"
+}
+
+print_command_list() {
+	echo "static struct cmdname_help command_list[] = {"
+
+	command_list "$1" |
+	while read cmd rest
+	do
+		printf "	{ \"$cmd\", $(get_synopsis $cmd), 0"
+		for cat in $(echo "$rest" | get_categories)
+		do
+			printf " | CAT_$cat"
+		done
+		echo " },"
+	done
+	echo "};"
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
-	char name[16];
-	char help[80];
-	unsigned char group;
+	const char *name;
+	const char *help;
+	uint32_t group;
 };
+"
+if [ -z "$2" ]
+then
+	define_categories "$1"
+	echo
+	print_command_list "$1"
+	exit 0
+fi
 
-static const char *common_cmd_groups[] = {"
+echo "static const char *common_cmd_groups[] = {"
 
 grps=grps$$.tmp
 match=match$$.tmp
-- 
2.17.0.519.gb89679a4aa
Eric Sunshine· Apr 25, 2018, 18:07 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v4/wip 02/12] generate-cmds.sh: export all commands to command-list.h

On Wed, Apr 25, 2018 at 12:30 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 22 quoted lines
> The current generate-cmds.sh generates just enough to print "git help"
> output. That is, it only extracts help text for common commands.
>
> The script is now updated to extract help text for all commands and
> keep command classification a new file, command-list.h. This will be
> useful later:
> [...]
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> @@ -12,14 +34,51 @@ get_synopsis () {
> +define_categories() {
> +       echo
> +       echo "/* Command categories */"
> +       bit=0
> +       category_list "$1" |
> +       while read cat
> +       do
> +               echo "#define CAT_$cat (1UL << $bit)"
> +               bit=$(($bit+1))
> +       done
> +       test "$bit" -gt 32 && die "Urgh.. too many categories?"
Should this be '-ge' rather than '-gt'?
> +}
Duy Nguyen· Apr 29, 2018, 16:11 UTC · re: Eric Sunshine · lore

Re: [PATCH v4/wip 02/12] generate-cmds.sh: export all commands to command-list.h

On Wed, Apr 25, 2018 at 8:07 PM, Eric Sunshine <sunshine@sunshineco.com> wrote:
Show 26 quoted lines
> On Wed, Apr 25, 2018 at 12:30 PM, Nguyễn Thái Ngọc Duy
> <pclouds@gmail.com> wrote:
>> The current generate-cmds.sh generates just enough to print "git help"
>> output. That is, it only extracts help text for common commands.
>>
>> The script is now updated to extract help text for all commands and
>> keep command classification a new file, command-list.h. This will be
>> useful later:
>> [...]
>> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
>> ---
>> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
>> @@ -12,14 +34,51 @@ get_synopsis () {
>> +define_categories() {
>> +       echo
>> +       echo "/* Command categories */"
>> +       bit=0
>> +       category_list "$1" |
>> +       while read cat
>> +       do
>> +               echo "#define CAT_$cat (1UL << $bit)"
>> +               bit=$(($bit+1))
>> +       done
>> +       test "$bit" -gt 32 && die "Urgh.. too many categories?"
>
> Should this be '-ge' rather than '-gt'?

After we print "1UL << 31" we increment it to 32 and exit the loop, then it's still within limits, -ge would incorrectly complain

-- 
Duy
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:30 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 03/12] help: use command-list.h for common command list

The previous commit added code generation for all_cmd_desc[] which includes almost everything we need to generate common command list. Convert help code to use that array instead and drop common_cmds[] array.

The description of each common command group is removed from command-list.txt. This keeps this file format simpler. common-cmds.h will not be generated correctly after this change due to the command-list.txt format change. But it does not matter and common-cmds.h will be removed.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Makefile            |   4 +-
 command-list.txt    |  10 ----
 generate-cmdlist.sh |   4 +-
 help.c              | 135 ++++++++++++++++++++++++++++++++------------
 4 files changed, 103 insertions(+), 50 deletions(-)
Show changes to 4 files +103 −50

Makefile, command-list.txt, generate-cmdlist.sh, help.c

diff --git a/Makefile b/Makefile
index 2a8913ea21..5c58b0b692 100644
--- a/Makefile
+++ b/Makefile
@@ -1914,9 +1914,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
-help.sp help.s help.o: common-cmds.h
+help.sp help.s help.o: common-cmds.h command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h command-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
diff --git a/command-list.txt b/command-list.txt
index 786536aba0..3bd23201a6 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -1,13 +1,3 @@
-# common commands are grouped by themes
-# these groups are output by 'git help' in the order declared here.
-# map each common command in the command list to one of these groups.
-### common groups (do not change this line)
-init         start a working area (see also: git help tutorial)
-worktree     work on the current change (see also: git help everyday)
-info         examine the history and state (see also: git help revisions)
-history      grow, mark and tweak your common history
-remote       collaborate (see also: git help workflows)
-
 ### command list (do not change this line, also do not change alignment)
 # command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 9ba9911f09..88968160e3 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -6,7 +6,7 @@ die () {
 }
 
 command_list () {
-	sed '1,/^### command list/d;/^#/d' "$1"
+	grep -v '^#' "$1"
 }
 
 get_categories() {
@@ -67,7 +67,7 @@ echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
 	const char *name;
 	const char *help;
-	uint32_t group;
+	uint32_t category;
 };
 "
 if [ -z "$2" ]
diff --git a/help.c b/help.c
index 60071a9bea..5b4a2f1b4f 100644
--- a/help.c
+++ b/help.c
@@ -5,13 +5,104 @@
 #include "run-command.h"
 #include "levenshtein.h"
 #include "help.h"
-#include "common-cmds.h"
+#include "command-list.h"
 #include "string-list.h"
 #include "column.h"
 #include "version.h"
 #include "refs.h"
 #include "parse-options.h"
 
+struct category_description {
+	uint32_t category;
+	const char *desc;
+};
+static uint32_t common_mask =
+	CAT_init | CAT_worktree | CAT_info |
+	CAT_history | CAT_remote;
+static struct category_description common_categories[] = {
+	{ CAT_init, N_("start a working area (see also: git help tutorial)") },
+	{ CAT_worktree, N_("work on the current change (see also: git help everyday)") },
+	{ CAT_info, N_("examine the history and state (see also: git help revisions)") },
+	{ CAT_history, N_("grow, mark and tweak your common history") },
+	{ CAT_remote, N_("collaborate (see also: git help workflows)") },
+	{ 0, NULL }
+};
+
+static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
+{
+	int i, nr = 0;
+	struct cmdname_help *cmds;
+
+	ALLOC_ARRAY(cmds, ARRAY_SIZE(command_list) + 1);
+
+	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
+		const struct cmdname_help *cmd = command_list + i;
+		const char *name;
+
+		if (!(cmd->category & mask))
+			continue;
+
+		cmds[nr] = *cmd;
+
+		if (skip_prefix(cmd->name, "git-", &name))
+			cmds[nr].name = name;
+
+		nr++;
+	}
+	cmds[nr].name = NULL;
+	*p_cmds = cmds;
+}
+
+static void print_command_list(const struct cmdname_help *cmds,
+			       uint32_t mask, int longest)
+{
+	int i;
+
+	for (i = 0; cmds[i].name; i++) {
+		if (cmds[i].category & mask) {
+			printf("   %s   ", cmds[i].name);
+			mput_char(' ', longest - strlen(cmds[i].name));
+			puts(_(cmds[i].help));
+		}
+	}
+}
+
+static int cmd_name_cmp(const void *elem1, const void *elem2)
+{
+	const struct cmdname_help *e1 = elem1;
+	const struct cmdname_help *e2 = elem2;
+
+	return strcmp(e1->name, e2->name);
+}
+
+static void print_cmd_by_category(const struct category_description *catdesc)
+{
+	struct cmdname_help *cmds;
+	int longest = 0;
+	int i, nr = 0;
+	uint32_t mask = 0;
+
+	for (i = 0; catdesc[i].desc; i++)
+		mask |= catdesc[i].category;
+
+	extract_cmds(&cmds, mask);
+
+	for (i = 0; cmds[i].name; i++, nr++) {
+		if (longest < strlen(cmds[i].name))
+			longest = strlen(cmds[i].name);
+	}
+	QSORT(cmds, nr, cmd_name_cmp);
+
+	for (i = 0; catdesc[i].desc; i++) {
+		uint32_t mask = catdesc[i].category;
+		const char *desc = catdesc[i].desc;
+
+		printf("\n%s\n", _(desc));
+		print_command_list(cmds, mask, longest);
+	}
+	free(cmds);
+}
+
 void add_cmdname(struct cmdnames *cmds, const char *name, int len)
 {
 	struct cmdname *ent;
@@ -190,42 +281,10 @@ void list_commands(unsigned int colopts,
 	}
 }
 
-static int cmd_group_cmp(const void *elem1, const void *elem2)
-{
-	const struct cmdname_help *e1 = elem1;
-	const struct cmdname_help *e2 = elem2;
-
-	if (e1->group < e2->group)
-		return -1;
-	if (e1->group > e2->group)
-		return 1;
-	return strcmp(e1->name, e2->name);
-}
-
 void list_common_cmds_help(void)
 {
-	int i, longest = 0;
-	int current_grp = -1;
-
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
-		if (longest < strlen(common_cmds[i].name))
-			longest = strlen(common_cmds[i].name);
-	}
-
-	QSORT(common_cmds, ARRAY_SIZE(common_cmds), cmd_group_cmp);
-
 	puts(_("These are common Git commands used in various situations:"));
-
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
-		if (common_cmds[i].group != current_grp) {
-			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
-			current_grp = common_cmds[i].group;
-		}
-
-		printf("   %s   ", common_cmds[i].name);
-		mput_char(' ', longest - strlen(common_cmds[i].name));
-		puts(_(common_cmds[i].help));
-	}
+	print_cmd_by_category(common_categories);
 }
 
 int is_in_cmdlist(struct cmdnames *c, const char *s)
@@ -285,6 +344,7 @@ const char *help_unknown_cmd(const char *cmd)
 {
 	int i, n, best_similarity = 0;
 	struct cmdnames main_cmds, other_cmds;
+	struct cmdname_help *common_cmds;
 
 	memset(&main_cmds, 0, sizeof(main_cmds));
 	memset(&other_cmds, 0, sizeof(other_cmds));
@@ -299,6 +359,8 @@ const char *help_unknown_cmd(const char *cmd)
 	QSORT(main_cmds.names, main_cmds.cnt, cmdname_compare);
 	uniq(&main_cmds);
 
+	extract_cmds(&common_cmds, common_mask);
+
 	/* This abuses cmdname->len for levenshtein distance */
 	for (i = 0, n = 0; i < main_cmds.cnt; i++) {
 		int cmp = 0; /* avoid compiler stupidity */
@@ -313,10 +375,10 @@ const char *help_unknown_cmd(const char *cmd)
 			die(_(bad_interpreter_advice), cmd, cmd);
 
 		/* Does the candidate appear in common_cmds list? */
-		while (n < ARRAY_SIZE(common_cmds) &&
+		while (common_cmds[n].name &&
 		       (cmp = strcmp(common_cmds[n].name, candidate)) < 0)
 			n++;
-		if ((n < ARRAY_SIZE(common_cmds)) && !cmp) {
+		if (common_cmds[n].name && !cmp) {
 			/* Yes, this is one of the common commands */
 			n++; /* use the entry from common_cmds[] */
 			if (starts_with(candidate, cmd)) {
@@ -329,6 +391,7 @@ const char *help_unknown_cmd(const char *cmd)
 		main_cmds.names[i]->len =
 			levenshtein(cmd, candidate, 0, 2, 1, 3) + 1;
 	}
+	FREE_AND_NULL(common_cmds);
 
 	QSORT(main_cmds.names, main_cmds.cnt, levenshtein_compare);
 
-- 
2.17.0.519.gb89679a4aa
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:31 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 08/12] git: support --list-cmds=<category>

This allows us to select any group of commands by a category defined in command-list.txt. This is an internal/hidden option so we don't have to be picky about the category name or worried about exposing too much.

This will be used later by git-completion.bash to retrieve certain command groups.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 generate-cmdlist.sh | 17 +++++++++++++++++
 git.c               |  2 +-
 help.c              | 19 +++++++++++++++++++
 help.h              |  1 +
 4 files changed, 38 insertions(+), 1 deletion(-)
Show changes to 4 files +38 −1

generate-cmdlist.sh, git.c, help.c, help.h

diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 57f9800123..c14c0a7adc 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -47,6 +47,21 @@ define_categories() {
 	test "$bit" -gt 32 && die "Urgh.. too many categories?"
 }
 
+define_category_names() {
+	echo
+	echo "/* Category names */"
+	echo "static const char *category_names[] = {"
+	bit=0
+	category_list "$1" |
+	while read cat
+	do
+		echo "	\"$cat\", /* (1UL << $bit) */"
+		bit=$(($bit+1))
+	done
+	echo "	NULL"
+	echo "};"
+}
+
 print_command_list() {
 	echo "static struct cmdname_help command_list[] = {"
 
@@ -72,4 +87,6 @@ struct cmdname_help {
 "
 define_categories "$1"
 echo
+define_category_names "$1"
+echo
 print_command_list "$1"
diff --git a/git.c b/git.c
index fa6e542d06..6c071d44f5 100644
--- a/git.c
+++ b/git.c
@@ -232,7 +232,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			else if (!strcmp(cmd, "all"))
 				list_all_cmds();
 			else
-				die("unsupported command listing type '%s'", cmd);
+				list_cmds_by_category(cmd);
 			commands_listed++;
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
diff --git a/help.c b/help.c
index 217864999e..affa8e4343 100644
--- a/help.c
+++ b/help.c
@@ -305,6 +305,25 @@ void list_all_cmds(void)
 	clean_cmdnames(&other_cmds);
 }
 
+void list_cmds_by_category(const char *cat)
+{
+	int i;
+	int cat_id = 0;
+
+	for (i = 0; category_names[i]; i++) {
+		if (!strcmp(cat, category_names[i])) {
+			cat_id = 1 << i;
+			break;
+		}
+	}
+	if (!cat_id)
+		die("unsupported command listing type '%s'", cat);
+
+	for (i = 0; command_list[i].name; i++)
+		if (command_list[i].category & cat_id)
+			puts(command_list[i].name);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 0bf29f8dc5..3e30542927 100644
--- a/help.h
+++ b/help.h
@@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds(void);
+extern void list_cmds_by_category(const char *category);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.519.gb89679a4aa
Eric Sunshine· Apr 25, 2018, 18:16 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v4/wip 08/12] git: support --list-cmds=<category>

On Wed, Apr 25, 2018 at 12:31 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 16 quoted lines
> This allows us to select any group of commands by a category defined
> in command-list.txt. This is an internal/hidden option so we don't
> have to be picky about the category name or worried about exposing too
> much.
>
> This will be used later by git-completion.bash to retrieve certain
> command groups.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
> diff --git a/help.c b/help.c
> @@ -305,6 +305,25 @@ void list_all_cmds(void)
> +void list_cmds_by_category(const char *cat)
> +{
> +       int i;
> +       int cat_id = 0;
Should 'cat_id' be unsigned...
> +
> +       for (i = 0; category_names[i]; i++) {
> +               if (!strcmp(cat, category_names[i])) {
> +                       cat_id = 1 << i;
...since you're shifting it here?
Show 10 quoted lines
> +                       break;
> +               }
> +       }
> +       if (!cat_id)
> +               die("unsupported command listing type '%s'", cat);
> +
> +       for (i = 0; command_list[i].name; i++)
> +               if (command_list[i].category & cat_id)
> +                       puts(command_list[i].name);
> +}
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:31 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 07/12] completion: implement and use --list-cmds=all

Instead of parsing "git help -a" output, which is tricky to get right, less elegant and also slow, make git provide the list in machine-friendly form.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  |  2 ++
 help.c                                 | 18 ++++++++++++++++++
 help.h                                 |  1 +
 4 files changed, 22 insertions(+), 1 deletion(-)
Show changes to 4 files +22 −1

contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3556838759..a5f13ade20 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -839,7 +839,7 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git help -a|egrep '^  [a-zA-Z0-9]'
+		git --list-cmds=all
 	fi
 }
 
diff --git a/git.c b/git.c
index a46263306d..fa6e542d06 100644
--- a/git.c
+++ b/git.c
@@ -229,6 +229,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 				list_builtins(0, '\n');
 			else if (!strcmp(cmd, "parseopt"))
 				list_builtins(NO_PARSEOPT, ' ');
+			else if (!strcmp(cmd, "all"))
+				list_all_cmds();
 			else
 				die("unsupported command listing type '%s'", cmd);
 			commands_listed++;
diff --git a/help.c b/help.c
index 5b4a2f1b4f..217864999e 100644
--- a/help.c
+++ b/help.c
@@ -287,6 +287,24 @@ void list_common_cmds_help(void)
 	print_cmd_by_category(common_categories);
 }
 
+void list_all_cmds(void)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < main_cmds.cnt; i++)
+		puts(main_cmds.names[i]->name);
+	for (i = 0; i < other_cmds.cnt; i++)
+		puts(other_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index b21d7c94e8..0bf29f8dc5 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.519.gb89679a4aa
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:31 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 06/12] git: accept multiple --list-cmds options

Later on we may support non-overlapping command groups to --list-cmds. Allow the user to execute just one "git" process and get multiple groups. This may matter for git-completion.bash on Windows because we don't want the user to way for long when TAB-ing and Windows is slow on launching new processes.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 git.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
Show changes to git.c +4 −1
diff --git a/git.c b/git.c
index 28bfa96d87..a46263306d 100644
--- a/git.c
+++ b/git.c
@@ -64,6 +64,7 @@ void setup_auto_pager(const char *cmd, int def)
 static int handle_options(const char ***argv, int *argc, int *envchanged)
 {
 	const char **orig_argv = *argv;
+	int commands_listed = 0;
 
 	while (*argc > 0) {
 		const char *cmd = (*argv)[0];
@@ -230,7 +231,7 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 				list_builtins(NO_PARSEOPT, ' ');
 			else
 				die("unsupported command listing type '%s'", cmd);
-			exit(0);
+			commands_listed++;
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
 			usage(git_usage_string);
@@ -239,6 +240,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 		(*argv)++;
 		(*argc)--;
 	}
+	if (commands_listed)
+		exit(0);
 	return (*argv) - orig_argv;
 }
 
-- 
2.17.0.519.gb89679a4aa
Eric Sunshine· Apr 25, 2018, 18:12 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v4/wip 06/12] git: accept multiple --list-cmds options

On Wed, Apr 25, 2018 at 12:31 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

> Later on we may support non-overlapping command groups to
> --list-cmds. Allow the user to execute just one "git" process and get
> multiple groups. This may matter for git-completion.bash on Windows
> because we don't want the user to way for long when TAB-ing and
s/way/wait/
> Windows is slow on launching new processes.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:31 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 11/12] command-list.txt: add new category "complete"

This category, combined with 'external' and 'mainporcelain', is intended to replace the "porcelain command list" in git-completion.bash. In other words, these are the commands that will show up by default when you type "git <tab>".

Compared to the current list in git-completion.bash (which is basically "git help -a" with a black list), the following commands no longer show up:

- annotate          obsolete, discouraged to use
- difftool-helper   not an end user command
- filter-branch     not often used
- get-tar-commit-id not often used
- imap-send         not often used
- instaweb          does anybody still run this interactively?
- interpreter-trailers not an interactive command
- lost-found        obsolete
- mergetool         ???
- p4                too short to complete?
- peek-remote       not often used??
- svn               ???
- tar-tree          obsolete, use archive instead
- verify-commit     not often used
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 command-list.txt | 33 +++++++++++++++++----------------
 1 file changed, 17 insertions(+), 16 deletions(-)
Show changes to command-list.txt +17 −16
diff --git a/command-list.txt b/command-list.txt
index a57bcb64a1..40b56c57f5 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -3,11 +3,11 @@
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
-git-apply                               plumbingmanipulators
+git-apply                               plumbingmanipulators            complete
 git-archimport                          foreignscminterface
 git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
-git-blame                               ancillaryinterrogators
+git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
@@ -17,7 +17,7 @@ git-check-mailmap                       purehelpers
 git-checkout                            mainporcelain           history
 git-checkout-index                      plumbingmanipulators
 git-check-ref-format                    purehelpers
-git-cherry                              ancillaryinterrogators
+git-cherry                              ancillaryinterrogators          complete
 git-cherry-pick                         mainporcelain
 git-citool                              mainporcelain
 git-clean                               mainporcelain
@@ -25,7 +25,7 @@ git-clone                               mainporcelain           init
 git-column                              purehelpers
 git-commit                              mainporcelain           history
 git-commit-tree                         plumbingmanipulators
-git-config                              ancillarymanipulators
+git-config                              ancillarymanipulators           complete
 git-count-objects                       ancillaryinterrogators
 git-credential                          purehelpers
 git-credential-cache                    purehelpers
@@ -39,7 +39,7 @@ git-diff                                mainporcelain           history
 git-diff-files                          plumbinginterrogators
 git-diff-index                          plumbinginterrogators
 git-diff-tree                           plumbinginterrogators
-git-difftool                            ancillaryinterrogators
+git-difftool                            ancillaryinterrogators          complete
 git-fast-export                         ancillarymanipulators
 git-fast-import                         ancillarymanipulators
 git-fetch                               mainporcelain           remote
@@ -48,13 +48,13 @@ git-filter-branch                       ancillarymanipulators
 git-fmt-merge-msg                       purehelpers
 git-for-each-ref                        plumbinginterrogators
 git-format-patch                        mainporcelain
-git-fsck                                ancillaryinterrogators
+git-fsck                                ancillaryinterrogators          complete
 git-gc                                  mainporcelain
 git-get-tar-commit-id                   ancillaryinterrogators
 git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
-git-help                                ancillaryinterrogators
+git-help                                ancillaryinterrogators          complete
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
@@ -80,7 +80,7 @@ git-merge-tree                          ancillaryinterrogators
 git-mktag                               plumbingmanipulators
 git-mktree                              plumbingmanipulators
 git-mv                                  mainporcelain           worktree
-git-name-rev                            plumbinginterrogators
+git-name-rev                            plumbinginterrogators           complete
 git-notes                               mainporcelain
 git-p4                                  foreignscminterface
 git-pack-objects                        plumbingmanipulators
@@ -96,28 +96,29 @@ git-quiltimport                         foreignscminterface
 git-read-tree                           plumbingmanipulators
 git-rebase                              mainporcelain           history
 git-receive-pack                        synchelpers
-git-reflog                              ancillarymanipulators
-git-remote                              ancillarymanipulators
-git-repack                              ancillarymanipulators
-git-replace                             ancillarymanipulators
-git-request-pull                        foreignscminterface
+git-reflog                              ancillarymanipulators           complete
+git-remote                              ancillarymanipulators           complete
+git-repack                              ancillarymanipulators           complete
+git-replace                             ancillarymanipulators           complete
+git-request-pull                        foreignscminterface             complete
 git-rerere                              ancillaryinterrogators
 git-reset                               mainporcelain           worktree
 git-revert                              mainporcelain
 git-rev-list                            plumbinginterrogators
 git-rev-parse                           ancillaryinterrogators
 git-rm                                  mainporcelain           worktree
-git-send-email                          foreignscminterface
+git-send-email                          foreignscminterface             complete
 git-send-pack                           synchingrepositories
 git-shell                               synchelpers
 git-shortlog                            mainporcelain
 git-show                                mainporcelain           info
-git-show-branch                         ancillaryinterrogators
+git-show-branch                         ancillaryinterrogators          complete
 git-show-index                          plumbinginterrogators
 git-show-ref                            plumbinginterrogators
 git-sh-i18n                             purehelpers
 git-sh-setup                            purehelpers
 git-stash                               mainporcelain
+git-stage                                                               complete
 git-status                              mainporcelain           info
 git-stripspace                          purehelpers
 git-submodule                           mainporcelain
@@ -136,7 +137,7 @@ git-verify-commit                       ancillaryinterrogators
 git-verify-pack                         plumbinginterrogators
 git-verify-tag                          ancillaryinterrogators
 gitweb                                  ancillaryinterrogators
-git-whatchanged                         ancillaryinterrogators
+git-whatchanged                         ancillaryinterrogators          complete
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
 gitattributes                           guide
-- 
2.17.0.519.gb89679a4aa
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:31 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 09/12] help: add "-a --verbose" to list all commands with synopsis

This lists all recognized commands [1] by category. The group order follows closely git.txt.

[1] We may actually show commands that are not built (e.g. if you set NO_PERL you don't have git-instaweb but it's still listed here). I ignore the problem because on Linux a git package could be split anyway. The "git-core" package may not contain git-instaweb even if it's built because it may end up in a separate package. We can't know anyway.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-help.txt |  4 +++-
 builtin/help.c             |  7 +++++++
 help.c                     | 16 ++++++++++++++++
 help.h                     |  1 +
 t/t0012-help.sh            |  9 +++++++++
 5 files changed, 36 insertions(+), 1 deletion(-)
Show changes to 5 files +36 −1

Documentation/git-help.txt, builtin/help.c, help.c, help.h, t/t0012-help.sh

diff --git a/Documentation/git-help.txt b/Documentation/git-help.txt
index 40d328a4b3..a40fc38d8b 100644
--- a/Documentation/git-help.txt
+++ b/Documentation/git-help.txt
@@ -8,7 +8,7 @@ git-help - Display help information about Git
 SYNOPSIS
 --------
 [verse]
-'git help' [-a|--all] [-g|--guide]
+'git help' [-a|--all [--verbose]] [-g|--guide]
 	   [-i|--info|-m|--man|-w|--web] [COMMAND|GUIDE]
 
 DESCRIPTION
@@ -42,6 +42,8 @@ OPTIONS
 --all::
 	Prints all the available commands on the standard output. This
 	option overrides any given command or guide name.
+	When used with `--verbose` print description for all recognized
+	commands.
 
 -g::
 --guides::
diff --git a/builtin/help.c b/builtin/help.c
index 598867cfea..0e0af8426a 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -36,6 +36,7 @@ static const char *html_path;
 
 static int show_all = 0;
 static int show_guides = 0;
+static int verbose;
 static unsigned int colopts;
 static enum help_format help_format = HELP_FORMAT_NONE;
 static int exclude_guides;
@@ -48,6 +49,7 @@ static struct option builtin_help_options[] = {
 			HELP_FORMAT_WEB),
 	OPT_SET_INT('i', "info", &help_format, N_("show info page"),
 			HELP_FORMAT_INFO),
+	OPT__VERBOSE(&verbose, N_("print command description")),
 	OPT_END(),
 };
 
@@ -463,6 +465,11 @@ int cmd_help(int argc, const char **argv, const char *prefix)
 
 	if (show_all) {
 		git_config(git_help_config, NULL);
+		if (verbose) {
+			setup_pager();
+			list_all_cmds_help();
+			return 0;
+		}
 		printf(_("usage: %s%s"), _(git_usage_string), "\n\n");
 		load_command_list("git-", &main_cmds, &other_cmds);
 		list_commands(colopts, &main_cmds, &other_cmds);
diff --git a/help.c b/help.c
index affa8e4343..1f27d94661 100644
--- a/help.c
+++ b/help.c
@@ -27,6 +27,17 @@ static struct category_description common_categories[] = {
 	{ CAT_remote, N_("collaborate (see also: git help workflows)") },
 	{ 0, NULL }
 };
+static struct category_description main_categories[] = {
+	{ CAT_mainporcelain, N_("Main Porcelain Commands") },
+	{ CAT_ancillarymanipulators, N_("Ancillary Commands / Manipulators") },
+	{ CAT_ancillaryinterrogators, N_("Ancillary Commands / Interrogators") },
+	{ CAT_foreignscminterface, N_("Interacting with Others") },
+	{ CAT_plumbingmanipulators, N_("Low-level Commands / Manipulators") },
+	{ CAT_plumbinginterrogators, N_("Low-level Commands / Interrogators") },
+	{ CAT_synchingrepositories, N_("Low-level Commands / Synching Repositories") },
+	{ CAT_purehelpers, N_("Low-level Commands / Internal Helpers") },
+	{ 0, NULL }
+};
 
 static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
 {
@@ -324,6 +335,11 @@ void list_cmds_by_category(const char *cat)
 			puts(command_list[i].name);
 }
 
+void list_all_cmds_help(void)
+{
+	print_cmd_by_category(main_categories);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 3e30542927..118b7d9947 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds_help(void);
 extern void list_all_cmds(void);
 extern void list_cmds_by_category(const char *category);
 extern const char *help_unknown_cmd(const char *cmd);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index fd2a7f27dc..53208ab20e 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -25,6 +25,15 @@ test_expect_success "setup" '
 	EOF
 '
 
+# make sure to exercise these code paths, the output is a bit tricky
+# to verify
+test_expect_success 'basic help commands' '
+	git help >/dev/null &&
+	git help -a >/dev/null &&
+	git help -g >/dev/null &&
+	git help -av >/dev/null
+'
+
 test_expect_success "works for commands and guides by default" '
 	configure_help &&
 	git help status &&
-- 
2.17.0.519.gb89679a4aa
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:31 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 10/12] help: use command-list.txt for the source of guides

The help command currently hard codes the list of guides and their summary in C. Let's move this list to command-list.txt. This lets us extract summary lines from Documentation/git*.txt. This also potentially lets us lists guides in git.txt, but I'll leave that for now.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/gitattributes.txt |  2 +-
 Documentation/gitmodules.txt    |  2 +-
 Documentation/gitrevisions.txt  |  2 +-
 Makefile                        |  2 +-
 builtin/help.c                  | 32 --------------------------------
 command-list.txt                |  8 ++++++++
 help.c                          | 14 +++++++++++++-
 help.h                          |  1 +
 8 files changed, 26 insertions(+), 37 deletions(-)
Show changes to 8 files +26 −37

Documentation/gitattributes.txt, Documentation/gitmodules.txt, Documentation/gitrevisions.txt, Makefile, builtin/help.c, command-list.txt, help.c, help.h

diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 1094fe2b5b..083c2f380d 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -3,7 +3,7 @@ gitattributes(5)
 
 NAME
 ----
-gitattributes - defining attributes per path
+gitattributes - Defining attributes per path
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index db5d47eb19..4d63def206 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -3,7 +3,7 @@ gitmodules(5)
 
 NAME
 ----
-gitmodules - defining submodule properties
+gitmodules - Defining submodule properties
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitrevisions.txt b/Documentation/gitrevisions.txt
index 27dec5b91d..1f6cceaefb 100644
--- a/Documentation/gitrevisions.txt
+++ b/Documentation/gitrevisions.txt
@@ -3,7 +3,7 @@ gitrevisions(7)
 
 NAME
 ----
-gitrevisions - specifying revisions and ranges for Git
+gitrevisions - Specifying revisions and ranges for Git
 
 SYNOPSIS
 --------
diff --git a/Makefile b/Makefile
index a60a78ee67..1efb751e46 100644
--- a/Makefile
+++ b/Makefile
@@ -1937,7 +1937,7 @@ $(BUILT_INS): git$X
 
 command-list.h: generate-cmdlist.sh command-list.txt
 
-command-list.h: $(wildcard Documentation/git-*.txt)
+command-list.h: $(wildcard Documentation/git*.txt)
 	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
diff --git a/builtin/help.c b/builtin/help.c
index 0e0af8426a..5727fb5e51 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -402,38 +402,6 @@ static void show_html_page(const char *git_cmd)
 	open_html(page_path.buf);
 }
 
-static struct {
-	const char *name;
-	const char *help;
-} common_guides[] = {
-	{ "attributes", N_("Defining attributes per path") },
-	{ "everyday", N_("Everyday Git With 20 Commands Or So") },
-	{ "glossary", N_("A Git glossary") },
-	{ "ignore", N_("Specifies intentionally untracked files to ignore") },
-	{ "modules", N_("Defining submodule properties") },
-	{ "revisions", N_("Specifying revisions and ranges for Git") },
-	{ "tutorial", N_("A tutorial introduction to Git (for version 1.5.1 or newer)") },
-	{ "workflows", N_("An overview of recommended workflows with Git") },
-};
-
-static void list_common_guides_help(void)
-{
-	int i, longest = 0;
-
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		if (longest < strlen(common_guides[i].name))
-			longest = strlen(common_guides[i].name);
-	}
-
-	puts(_("The common Git guides are:\n"));
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		printf("   %s   ", common_guides[i].name);
-		mput_char(' ', longest - strlen(common_guides[i].name));
-		puts(_(common_guides[i].help));
-	}
-	putchar('\n');
-}
-
 static const char *check_git_cmd(const char* cmd)
 {
 	char *alias;
diff --git a/command-list.txt b/command-list.txt
index 3bd23201a6..a57bcb64a1 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -139,3 +139,11 @@ gitweb                                  ancillaryinterrogators
 git-whatchanged                         ancillaryinterrogators
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
+gitattributes                           guide
+giteveryday                             guide
+gitglossary                             guide
+gitignore                               guide
+gitmodules                              guide
+gitrevisions                            guide
+gittutorial                             guide
+gitworkflows                            guide
diff --git a/help.c b/help.c
index 1f27d94661..d5a2aff4cb 100644
--- a/help.c
+++ b/help.c
@@ -55,7 +55,9 @@ static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
 
 		cmds[nr] = *cmd;
 
-		if (skip_prefix(cmd->name, "git-", &name))
+		if (skip_prefix(cmd->name, "git-", &name) ||
+		    (mask == CAT_guide &&
+		     skip_prefix(cmd->name, "git", &name)))
 			cmds[nr].name = name;
 
 		nr++;
@@ -335,6 +337,16 @@ void list_cmds_by_category(const char *cat)
 			puts(command_list[i].name);
 }
 
+void list_common_guides_help(void)
+{
+	struct category_description catdesc[] = {
+		{ CAT_guide, N_("The common Git guides are:") },
+		{ 0, NULL }
+	};
+	print_cmd_by_category(catdesc);
+	putchar('\n');
+}
+
 void list_all_cmds_help(void)
 {
 	print_cmd_by_category(main_categories);
diff --git a/help.h b/help.h
index 118b7d9947..71b1542896 100644
--- a/help.h
+++ b/help.h
@@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds_help(void);
+extern void list_common_guides_help(void);
 extern void list_all_cmds(void);
 extern void list_cmds_by_category(const char *category);
 extern const char *help_unknown_cmd(const char *cmd);
-- 
2.17.0.519.gb89679a4aa
Eric Sunshine· Apr 25, 2018, 18:22 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v4/wip 10/12] help: use command-list.txt for the source of guides

On Wed, Apr 25, 2018 at 12:31 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

> The help command currently hard codes the list of guides and their
> summary in C. Let's move this list to command-list.txt. This lets us
> extract summary lines from Documentation/git*.txt. This also
> potentially lets us lists guides in git.txt, but I'll leave that for
s/lists/list/
> now.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:31 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 05/12] git.c: convert --list-*builtins to --list-cmds=*

Even if these are hidden options, let's make them a bit more generic since we're introducing more listing types shortly.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  | 12 +++++++-----
 t/t0012-help.sh                        |  2 +-
 3 files changed, 9 insertions(+), 7 deletions(-)
Show changes to 3 files +9 −7

contrib/completion/git-completion.bash, git.c, t/t0012-help.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a757073945..3556838759 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -3049,7 +3049,7 @@ __git_complete_common () {
 __git_cmds_with_parseopt_helper=
 __git_support_parseopt_helper () {
 	test -n "$__git_cmds_with_parseopt_helper" ||
-		__git_cmds_with_parseopt_helper="$(__git --list-parseopt-builtins)"
+		__git_cmds_with_parseopt_helper="$(__git --list-cmds=parseopt)"
 
 	case " $__git_cmds_with_parseopt_helper " in
 	*" $1 "*)
diff --git a/git.c b/git.c
index 3a89893712..28bfa96d87 100644
--- a/git.c
+++ b/git.c
@@ -223,11 +223,13 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			}
 			(*argv)++;
 			(*argc)--;
-		} else if (!strcmp(cmd, "--list-builtins")) {
-			list_builtins(0, '\n');
-			exit(0);
-		} else if (!strcmp(cmd, "--list-parseopt-builtins")) {
-			list_builtins(NO_PARSEOPT, ' ');
+		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
+			if (!strcmp(cmd, "builtins"))
+				list_builtins(0, '\n');
+			else if (!strcmp(cmd, "parseopt"))
+				list_builtins(NO_PARSEOPT, ' ');
+			else
+				die("unsupported command listing type '%s'", cmd);
 			exit(0);
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 487b92a5de..fd2a7f27dc 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -50,7 +50,7 @@ test_expect_success "--help does not work for guides" "
 "
 
 test_expect_success 'generate builtin list' '
-	git --list-builtins >builtins
+	git --list-cmds=builtins >builtins
 '
 
 while read builtin
-- 
2.17.0.519.gb89679a4aa
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:30 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 04/12] Remove common-cmds.h

After the last patch, common-cmds.h is no longer used (and it was actually broken). Remove all related code. command-list.h will take its place from now on.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 .gitignore          |  1 -
 Makefile            | 17 ++++++-----------
 generate-cmdlist.sh | 46 +++------------------------------------------
 3 files changed, 9 insertions(+), 55 deletions(-)
Show changes to 3 files +9 −55

.gitignore, Makefile, generate-cmdlist.sh

diff --git a/.gitignore b/.gitignore
index d4c3914167..0836083992 100644
--- a/.gitignore
+++ b/.gitignore
@@ -179,7 +179,6 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
-/common-cmds.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 5c58b0b692..a60a78ee67 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h command-list.h
+GENERATED_H += command-list.h
 
 LIB_H = $(shell $(FIND) . \
 	-name .git -prune -o \
@@ -1914,9 +1914,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
-help.sp help.s help.o: common-cmds.h command-list.h
+help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -1935,11 +1935,6 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
-common-cmds.h: generate-cmdlist.sh command-list.txt
-
-common-cmds.h: $(wildcard Documentation/git-*.txt)
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON >$@+ && mv $@+ $@
-
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git-*.txt)
@@ -2153,7 +2148,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h or command-list.h
+# Dependencies on automatically generated headers such as command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2532,7 +2527,7 @@ sparse: $(SP_OBJ)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h command-list.h
+check: command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -2780,7 +2775,7 @@ clean: profile-clean coverage-clean
 	$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 88968160e3..57f9800123 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -70,46 +70,6 @@ struct cmdname_help {
 	uint32_t category;
 };
 "
-if [ -z "$2" ]
-then
-	define_categories "$1"
-	echo
-	print_command_list "$1"
-	exit 0
-fi
-
-echo "static const char *common_cmd_groups[] = {"
-
-grps=grps$$.tmp
-match=match$$.tmp
-trap "rm -f '$grps' '$match'" 0 1 2 3 15
-
-sed -n '
-	1,/^### common groups/b
-	/^### command list/q
-	/^#/b
-	/^[ 	]*$/b
-	h;s/^[^ 	][^ 	]*[ 	][ 	]*\(.*\)/	N_("\1"),/p
-	g;s/^\([^ 	][^ 	]*\)[ 	].*/\1/w '$grps'
-	' "$1"
-printf '};\n\n'
-
-n=0
-substnum=
-while read grp
-do
-	echo "^git-..*[ 	]$grp"
-	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
-	n=$(($n+1))
-done <"$grps" >"$match"
-
-printf 'static struct cmdname_help common_cmds[] = {\n'
-grep -f "$match" "$1" |
-sed 's/^git-//' |
-sort |
-while read cmd tags
-do
-	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
-	echo "	{\"$cmd\", $(get_synopsis git-$cmd), $tag},"
-done
-echo "};"
+define_categories "$1"
+echo
+print_command_list "$1"
-- 
2.17.0.519.gb89679a4aa
Nguyễn Thái Ngọc Duy· Apr 25, 2018, 16:31 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v4/wip 12/12] completion: let git provide the completable command list

Instead of maintaining a separate list of command classification, which often could go out of date, let's centralize the information back in git.

Note that the current completion script incorrectly classifies filter-branch as porcelain and t9902 tests this behavior. We keep it this way in t9902 because this test does not really care which particular command is porcelain or plubmbing, they're just names.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash | 119 +++++--------------------
 t/t9902-completion.sh                  |   5 +-
 2 files changed, 25 insertions(+), 99 deletions(-)
Show changes to 2 files +25 −99

contrib/completion/git-completion.bash, t/t9902-completion.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a5f13ade20..c74d1d7684 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -835,18 +835,30 @@ __git_complete_strategy ()
 }
 
 __git_commands () {
-	if test -n "${GIT_TESTING_COMMAND_COMPLETION:-}"
-	then
-		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
-	else
-		git --list-cmds=all
-	fi
+	case "$1" in
+	porcelain)
+		if test -n "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		then
+			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		else
+			git --list-cmds=mainporcelain --list-cmds=complete
+		fi
+		;;
+	all)
+		if test -n "$GIT_TESTING_ALL_COMMAND_LIST"
+		then
+			printf "%s" "$GIT_TESTING_ALL_COMMAND_LIST";;
+		else
+			git --list-cmds=all
+		fi
+		;;
+	esac
 }
 
-__git_list_all_commands ()
+__git_list_commands ()
 {
 	local i IFS=" "$'\n'
-	for i in $(__git_commands)
+	for i in $(__git_commands $1)
 	do
 		case $i in
 		*--*)             : helper pattern;;
@@ -859,101 +871,14 @@ __git_all_commands=
 __git_compute_all_commands ()
 {
 	test -n "$__git_all_commands" ||
-	__git_all_commands=$(__git_list_all_commands)
-}
-
-__git_list_porcelain_commands ()
-{
-	local i IFS=" "$'\n'
-	__git_compute_all_commands
-	for i in $__git_all_commands
-	do
-		case $i in
-		*--*)             : helper pattern;;
-		applymbox)        : ask gittus;;
-		applypatch)       : ask gittus;;
-		archimport)       : import;;
-		cat-file)         : plumbing;;
-		check-attr)       : plumbing;;
-		check-ignore)     : plumbing;;
-		check-mailmap)    : plumbing;;
-		check-ref-format) : plumbing;;
-		checkout-index)   : plumbing;;
-		column)           : internal helper;;
-		commit-tree)      : plumbing;;
-		count-objects)    : infrequent;;
-		credential)       : credentials;;
-		credential-*)     : credentials helper;;
-		cvsexportcommit)  : export;;
-		cvsimport)        : import;;
-		cvsserver)        : daemon;;
-		daemon)           : daemon;;
-		diff-files)       : plumbing;;
-		diff-index)       : plumbing;;
-		diff-tree)        : plumbing;;
-		fast-import)      : import;;
-		fast-export)      : export;;
-		fsck-objects)     : plumbing;;
-		fetch-pack)       : plumbing;;
-		fmt-merge-msg)    : plumbing;;
-		for-each-ref)     : plumbing;;
-		hash-object)      : plumbing;;
-		http-*)           : transport;;
-		index-pack)       : plumbing;;
-		init-db)          : deprecated;;
-		local-fetch)      : plumbing;;
-		ls-files)         : plumbing;;
-		ls-remote)        : plumbing;;
-		ls-tree)          : plumbing;;
-		mailinfo)         : plumbing;;
-		mailsplit)        : plumbing;;
-		merge-*)          : plumbing;;
-		mktree)           : plumbing;;
-		mktag)            : plumbing;;
-		pack-objects)     : plumbing;;
-		pack-redundant)   : plumbing;;
-		pack-refs)        : plumbing;;
-		parse-remote)     : plumbing;;
-		patch-id)         : plumbing;;
-		prune)            : plumbing;;
-		prune-packed)     : plumbing;;
-		quiltimport)      : import;;
-		read-tree)        : plumbing;;
-		receive-pack)     : plumbing;;
-		remote-*)         : transport;;
-		rerere)           : plumbing;;
-		rev-list)         : plumbing;;
-		rev-parse)        : plumbing;;
-		runstatus)        : plumbing;;
-		sh-setup)         : internal;;
-		shell)            : daemon;;
-		show-ref)         : plumbing;;
-		send-pack)        : plumbing;;
-		show-index)       : plumbing;;
-		ssh-*)            : transport;;
-		stripspace)       : plumbing;;
-		symbolic-ref)     : plumbing;;
-		unpack-file)      : plumbing;;
-		unpack-objects)   : plumbing;;
-		update-index)     : plumbing;;
-		update-ref)       : plumbing;;
-		update-server-info) : daemon;;
-		upload-archive)   : plumbing;;
-		upload-pack)      : plumbing;;
-		write-tree)       : plumbing;;
-		var)              : infrequent;;
-		verify-pack)      : infrequent;;
-		verify-tag)       : plumbing;;
-		*) echo $i;;
-		esac
-	done
+	__git_all_commands=$(__git_list_commands all)
 }
 
 __git_porcelain_commands=
 __git_compute_porcelain_commands ()
 {
 	test -n "$__git_porcelain_commands" ||
-	__git_porcelain_commands=$(__git_list_porcelain_commands)
+	__git_porcelain_commands=$(__git_list_commands porcelain)
 }
 
 # Lists all set config variables starting with the given section prefix,
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 1b34caa1e1..2f16679380 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -13,7 +13,7 @@ complete ()
 	return 0
 }
 
-# Be careful when updating this list:
+# Be careful when updating these lists:
 #
 # (1) The build tree may have build artifact from different branch, or
 #     the user's $PATH may have a random executable that may begin
@@ -30,7 +30,8 @@ complete ()
 #     completion for "git <TAB>", and a plumbing is excluded.  "add",
 #     "filter-branch" and "ls-files" are listed for this.
 
-GIT_TESTING_COMMAND_COMPLETION='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_ALL_COMMAND_LIST='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_PORCELAIN_COMMAND_LIST='add checkout filter-branch'
 
 . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash"
 
-- 
2.17.0.519.gb89679a4aa
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 00/10] Keep all info in command-list.txt in git binary

I think v5 is getting close to what I wanted to achieve from the RFC version (I skip v4 since I sent out v4/wip and another v4 may confuse people).

Interdiff is too large to be helpful, but the summary of changes compared to v3 is:

- common-cmds.h is renamed to command-list.h
- the common group description is moved from command-list.txt to
  help.c to simplify command-list.txt format
- generate-cmds.sh supports multiple categories per command, a new one
  "complete" is added to aid git-completion.bash
- multiple --list-cmds options is replaced with
  --list-cmds=<type>[,<type>...]. This allows easier group
  customization in git-completion.bash (not happens yet)
- __git_list_all_commands() and __git_list_porcelain_commands() for
  backward compatibility
- "git help <tab>" completion also makes use of guide list in
  command-list.txt
- better tests from Ramsay

There is one sticky point yet about the guides. I'll pull Phillip in and explain more in the relevant patch.

Nguyễn Thái Ngọc Duy (10):
  generate-cmds.sh: factor out synopsis extract code
  generate-cmds.sh: export all commands to command-list.h
  help: use command-list.h for common command list
  Remove common-cmds.h
  git.c: convert --list-*builtins to --list-cmds=*
  completion: implement and use --list-cmds=main,others
  git: support --list-cmds=list-<category>
  help: add "-a --verbose" to list all commands with synopsis
  help: use command-list.txt for the source of guides
  completion: let git provide the completable command list
 .gitignore                             |   2 +-
 Documentation/git-help.txt             |   4 +-
 Documentation/gitattributes.txt        |   2 +-
 Documentation/gitmodules.txt           |   2 +-
 Documentation/gitrevisions.txt         |   2 +-
 Makefile                               |  16 +-
 builtin/help.c                         |  39 +----
 command-list.txt                       |  67 ++++----
 contrib/completion/git-completion.bash | 134 +++++-----------
 generate-cmdlist.sh                    | 126 ++++++++++-----
 git.c                                  |  38 ++++-
 help.c                                 | 209 +++++++++++++++++++++----
 help.h                                 |   5 +
 t/t0012-help.sh                        |  26 ++-
 t/t9902-completion.sh                  |   5 +-
 15 files changed, 426 insertions(+), 251 deletions(-)
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 01/10] generate-cmds.sh: factor out synopsis extract code

This makes it easier to reuse the same code in another place (very soon).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 generate-cmdlist.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
Show changes to generate-cmdlist.sh +11 −7
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index eeea4b67ea..31b6d886cb 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,15 @@
 #!/bin/sh
 
+get_synopsis () {
+	sed -n '
+		/^NAME/,/'"$1"'/H
+		${
+			x
+			s/.*'"$1"' - \(.*\)/N_("\1")/
+			p
+		}' "Documentation/$1.txt"
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
 	char name[16];
@@ -39,12 +49,6 @@ sort |
 while read cmd tags
 do
 	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
-	sed -n '
-		/^NAME/,/git-'"$cmd"'/H
-		${
-			x
-			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
-			p
-		}' "Documentation/git-$cmd.txt"
+	echo "	{\"$cmd\", $(get_synopsis git-$cmd), $tag},"
 done
 echo "};"
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 02/10] generate-cmds.sh: export all commands to command-list.h

The current generate-cmds.sh generates just enough to print "git help" output. That is, it only extracts help text for common commands.

The script is now updated to extract help text for all commands and keep command classification a new file, command-list.h. This will be useful later:

- "git help -a" could print a short summary of all commands instead of
  just the common ones.
- "git" could produce a list of commands of one or more category. One
  of its use is to reduce another command classification embedded in
  git-completion.bash.

The new file can be generated but is not used anywhere yet. The plan is we migrate away from common-cmds.h. Then we can kill off common-cmds.h build rules and generation code (and also delete duplicate content in command-list.h which we keep for now to not mess generate-cmds.sh up too much).

PS. The new fixed column requirement on command-list.txt is technically not needed. But it helps simplify the code a bit at this stage. We could lift this restriction later if we want to.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 .gitignore          |  1 +
 Makefile            | 13 ++++++---
 command-list.txt    |  4 +--
 generate-cmdlist.sh | 67 ++++++++++++++++++++++++++++++++++++++++++---
 4 files changed, 75 insertions(+), 10 deletions(-)
Show changes to 4 files +75 −10

.gitignore, Makefile, command-list.txt, generate-cmdlist.sh

diff --git a/.gitignore b/.gitignore
index 833ef3b0b7..d4c3914167 100644
--- a/.gitignore
+++ b/.gitignore
@@ -180,6 +180,7 @@
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
 /common-cmds.h
+/command-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index 50da82b016..bb29470f88 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h
+GENERATED_H += common-cmds.h command-list.h
 
 LIB_H = $(shell $(FIND) . \
 	-name .git -prune -o \
@@ -1940,6 +1940,11 @@ $(BUILT_INS): git$X
 common-cmds.h: generate-cmdlist.sh command-list.txt
 
 common-cmds.h: $(wildcard Documentation/git-*.txt)
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON >$@+ && mv $@+ $@
+
+command-list.h: generate-cmdlist.sh command-list.txt
+
+command-list.h: $(wildcard Documentation/git-*.txt)
 	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
@@ -2150,7 +2155,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h
+# Dependencies on automatically generated headers such as common-cmds.h or command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2529,7 +2534,7 @@ sparse: $(SP_OBJ)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h
+check: common-cmds.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -2777,7 +2782,7 @@ clean: profile-clean coverage-clean
 	$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/command-list.txt b/command-list.txt
index a1fad28fd8..786536aba0 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -8,8 +8,8 @@ info         examine the history and state (see also: git help revisions)
 history      grow, mark and tweak your common history
 remote       collaborate (see also: git help workflows)
 
-### command list (do not change this line)
-# command name                          category [deprecated] [common]
+### command list (do not change this line, also do not change alignment)
+# command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 31b6d886cb..c9fd524760 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,27 @@
 #!/bin/sh
 
+die () {
+	echo "$@" >&2
+	exit 1
+}
+
+command_list () {
+	sed '1,/^### command list/d;/^#/d' "$1"
+}
+
+get_categories() {
+	tr ' ' '\n'|
+	grep -v '^$' |
+	sort |
+	uniq
+}
+
+category_list () {
+	command_list "$1" |
+	cut -c 40- |
+	get_categories
+}
+
 get_synopsis () {
 	sed -n '
 		/^NAME/,/'"$1"'/H
@@ -10,14 +32,51 @@ get_synopsis () {
 		}' "Documentation/$1.txt"
 }
 
+define_categories() {
+	echo
+	echo "/* Command categories */"
+	bit=0
+	category_list "$1" |
+	while read cat
+	do
+		echo "#define CAT_$cat (1UL << $bit)"
+		bit=$(($bit+1))
+	done
+	test "$bit" -gt 32 && die "Urgh.. too many categories?"
+}
+
+print_command_list() {
+	echo "static struct cmdname_help command_list[] = {"
+
+	command_list "$1" |
+	while read cmd rest
+	do
+		printf "	{ \"$cmd\", $(get_synopsis $cmd), 0"
+		for cat in $(echo "$rest" | get_categories)
+		do
+			printf " | CAT_$cat"
+		done
+		echo " },"
+	done
+	echo "};"
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
-	char name[16];
-	char help[80];
-	unsigned char group;
+	const char *name;
+	const char *help;
+	uint32_t group;
 };
+"
+if [ -z "$2" ]
+then
+	define_categories "$1"
+	echo
+	print_command_list "$1"
+	exit 0
+fi
 
-static const char *common_cmd_groups[] = {"
+echo "static const char *common_cmd_groups[] = {"
 
 grps=grps$$.tmp
 match=match$$.tmp
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 03/10] help: use command-list.h for common command list

The previous commit added code generation for all_cmd_desc[] which includes almost everything we need to generate common command list. Convert help code to use that array instead and drop common_cmds[] array.

The description of each common command group is removed from command-list.txt. This keeps this file format simpler. common-cmds.h will not be generated correctly after this change due to the command-list.txt format change. But it does not matter and common-cmds.h will be removed.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Makefile            |   4 +-
 command-list.txt    |  10 ---
 generate-cmdlist.sh |   4 +-
 help.c              | 145 +++++++++++++++++++++++++++++++++-----------
 t/t0012-help.sh     |   9 +++
 5 files changed, 122 insertions(+), 50 deletions(-)
Show changes to 5 files +122 −50

Makefile, command-list.txt, generate-cmdlist.sh, help.c, t/t0012-help.sh

diff --git a/Makefile b/Makefile
index bb29470f88..ab67150e68 100644
--- a/Makefile
+++ b/Makefile
@@ -1916,9 +1916,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
-help.sp help.s help.o: common-cmds.h
+help.sp help.s help.o: common-cmds.h command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h command-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
diff --git a/command-list.txt b/command-list.txt
index 786536aba0..3bd23201a6 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -1,13 +1,3 @@
-# common commands are grouped by themes
-# these groups are output by 'git help' in the order declared here.
-# map each common command in the command list to one of these groups.
-### common groups (do not change this line)
-init         start a working area (see also: git help tutorial)
-worktree     work on the current change (see also: git help everyday)
-info         examine the history and state (see also: git help revisions)
-history      grow, mark and tweak your common history
-remote       collaborate (see also: git help workflows)
-
 ### command list (do not change this line, also do not change alignment)
 # command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index c9fd524760..93de8e8f59 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -6,7 +6,7 @@ die () {
 }
 
 command_list () {
-	sed '1,/^### command list/d;/^#/d' "$1"
+	grep -v '^#' "$1"
 }
 
 get_categories() {
@@ -65,7 +65,7 @@ echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
 	const char *name;
 	const char *help;
-	uint32_t group;
+	uint32_t category;
 };
 "
 if [ -z "$2" ]
diff --git a/help.c b/help.c
index a4feef2ffe..bf2738e9ef 100644
--- a/help.c
+++ b/help.c
@@ -5,13 +5,114 @@
 #include "run-command.h"
 #include "levenshtein.h"
 #include "help.h"
-#include "common-cmds.h"
+#include "command-list.h"
 #include "string-list.h"
 #include "column.h"
 #include "version.h"
 #include "refs.h"
 #include "parse-options.h"
 
+struct category_description {
+	uint32_t category;
+	const char *desc;
+};
+static uint32_t common_mask =
+	CAT_init | CAT_worktree | CAT_info |
+	CAT_history | CAT_remote;
+static struct category_description common_categories[] = {
+	{ CAT_init, N_("start a working area (see also: git help tutorial)") },
+	{ CAT_worktree, N_("work on the current change (see also: git help everyday)") },
+	{ CAT_info, N_("examine the history and state (see also: git help revisions)") },
+	{ CAT_history, N_("grow, mark and tweak your common history") },
+	{ CAT_remote, N_("collaborate (see also: git help workflows)") },
+	{ 0, NULL }
+};
+
+static const char *drop_prefix(const char *name)
+{
+	const char *new_name;
+
+	if (skip_prefix(name, "git-", &new_name))
+		return new_name;
+	return name;
+
+}
+
+static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
+{
+	int i, nr = 0;
+	struct cmdname_help *cmds;
+
+	if (ARRAY_SIZE(command_list) == 0)
+		BUG("empty command_list[] is a sign of broken generate-cmdlist.sh");
+
+	ALLOC_ARRAY(cmds, ARRAY_SIZE(command_list) + 1);
+
+	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
+		const struct cmdname_help *cmd = command_list + i;
+
+		if (!(cmd->category & mask))
+			continue;
+
+		cmds[nr] = *cmd;
+		cmds[nr].name = drop_prefix(cmd->name);
+
+		nr++;
+	}
+	cmds[nr].name = NULL;
+	*p_cmds = cmds;
+}
+
+static void print_command_list(const struct cmdname_help *cmds,
+			       uint32_t mask, int longest)
+{
+	int i;
+
+	for (i = 0; cmds[i].name; i++) {
+		if (cmds[i].category & mask) {
+			printf("   %s   ", cmds[i].name);
+			mput_char(' ', longest - strlen(cmds[i].name));
+			puts(_(cmds[i].help));
+		}
+	}
+}
+
+static int cmd_name_cmp(const void *elem1, const void *elem2)
+{
+	const struct cmdname_help *e1 = elem1;
+	const struct cmdname_help *e2 = elem2;
+
+	return strcmp(e1->name, e2->name);
+}
+
+static void print_cmd_by_category(const struct category_description *catdesc)
+{
+	struct cmdname_help *cmds;
+	int longest = 0;
+	int i, nr = 0;
+	uint32_t mask = 0;
+
+	for (i = 0; catdesc[i].desc; i++)
+		mask |= catdesc[i].category;
+
+	extract_cmds(&cmds, mask);
+
+	for (i = 0; cmds[i].name; i++, nr++) {
+		if (longest < strlen(cmds[i].name))
+			longest = strlen(cmds[i].name);
+	}
+	QSORT(cmds, nr, cmd_name_cmp);
+
+	for (i = 0; catdesc[i].desc; i++) {
+		uint32_t mask = catdesc[i].category;
+		const char *desc = catdesc[i].desc;
+
+		printf("\n%s\n", _(desc));
+		print_command_list(cmds, mask, longest);
+	}
+	free(cmds);
+}
+
 void add_cmdname(struct cmdnames *cmds, const char *name, int len)
 {
 	struct cmdname *ent;
@@ -190,42 +291,10 @@ void list_commands(unsigned int colopts,
 	}
 }
 
-static int cmd_group_cmp(const void *elem1, const void *elem2)
-{
-	const struct cmdname_help *e1 = elem1;
-	const struct cmdname_help *e2 = elem2;
-
-	if (e1->group < e2->group)
-		return -1;
-	if (e1->group > e2->group)
-		return 1;
-	return strcmp(e1->name, e2->name);
-}
-
 void list_common_cmds_help(void)
 {
-	int i, longest = 0;
-	int current_grp = -1;
-
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
-		if (longest < strlen(common_cmds[i].name))
-			longest = strlen(common_cmds[i].name);
-	}
-
-	QSORT(common_cmds, ARRAY_SIZE(common_cmds), cmd_group_cmp);
-
 	puts(_("These are common Git commands used in various situations:"));
-
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
-		if (common_cmds[i].group != current_grp) {
-			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
-			current_grp = common_cmds[i].group;
-		}
-
-		printf("   %s   ", common_cmds[i].name);
-		mput_char(' ', longest - strlen(common_cmds[i].name));
-		puts(_(common_cmds[i].help));
-	}
+	print_cmd_by_category(common_categories);
 }
 
 int is_in_cmdlist(struct cmdnames *c, const char *s)
@@ -285,6 +354,7 @@ const char *help_unknown_cmd(const char *cmd)
 {
 	int i, n, best_similarity = 0;
 	struct cmdnames main_cmds, other_cmds;
+	struct cmdname_help *common_cmds;
 
 	memset(&main_cmds, 0, sizeof(main_cmds));
 	memset(&other_cmds, 0, sizeof(other_cmds));
@@ -299,6 +369,8 @@ const char *help_unknown_cmd(const char *cmd)
 	QSORT(main_cmds.names, main_cmds.cnt, cmdname_compare);
 	uniq(&main_cmds);
 
+	extract_cmds(&common_cmds, common_mask);
+
 	/* This abuses cmdname->len for levenshtein distance */
 	for (i = 0, n = 0; i < main_cmds.cnt; i++) {
 		int cmp = 0; /* avoid compiler stupidity */
@@ -313,10 +385,10 @@ const char *help_unknown_cmd(const char *cmd)
 			die(_(bad_interpreter_advice), cmd, cmd);
 
 		/* Does the candidate appear in common_cmds list? */
-		while (n < ARRAY_SIZE(common_cmds) &&
+		while (common_cmds[n].name &&
 		       (cmp = strcmp(common_cmds[n].name, candidate)) < 0)
 			n++;
-		if ((n < ARRAY_SIZE(common_cmds)) && !cmp) {
+		if (common_cmds[n].name && !cmp) {
 			/* Yes, this is one of the common commands */
 			n++; /* use the entry from common_cmds[] */
 			if (starts_with(candidate, cmd)) {
@@ -329,6 +401,7 @@ const char *help_unknown_cmd(const char *cmd)
 		main_cmds.names[i]->len =
 			levenshtein(cmd, candidate, 0, 2, 1, 3) + 1;
 	}
+	FREE_AND_NULL(common_cmds);
 
 	QSORT(main_cmds.names, main_cmds.cnt, levenshtein_compare);
 
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 487b92a5de..c096f33505 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -49,6 +49,15 @@ test_expect_success "--help does not work for guides" "
 	test_i18ncmp expect actual
 "
 
+test_expect_success 'git help' '
+	git help >help.output &&
+	test_i18ngrep "^   clone  " help.output &&
+	test_i18ngrep "^   add    " help.output &&
+	test_i18ngrep "^   log    " help.output &&
+	test_i18ngrep "^   commit " help.output &&
+	test_i18ngrep "^   fetch  " help.output
+'
+
 test_expect_success 'generate builtin list' '
 	git --list-builtins >builtins
 '
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 06/10] completion: implement and use --list-cmds=main,others

Instead of parsing "git help -a" output, which is tricky to get right, less elegant and also slow, make git provide the list in a machine-friendly form. This adds two separate listing types, main and others, instead of just "all" for more flexibility.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  |  4 ++++
 help.c                                 | 32 ++++++++++++++++++++++++++
 help.h                                 |  2 ++
 4 files changed, 39 insertions(+), 1 deletion(-)
Show changes to 4 files +39 −1

contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index d7b448fd94..77cfb8a20b 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -834,7 +834,7 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git help -a|egrep '^  [a-zA-Z0-9]'
+		git --list-cmds=main,others
 	fi
 }
 
diff --git a/git.c b/git.c
index ea9bbfb6a3..da161a74ae 100644
--- a/git.c
+++ b/git.c
@@ -48,6 +48,10 @@ static int list_cmds(const char *spec)
 			list_builtins(0, '\n');
 		else if (len == 8 && !strncmp(spec, "parseopt", 8))
 			list_builtins(NO_PARSEOPT, ' ');
+		else if (len == 4 && !strncmp(spec, "main", 4))
+			list_all_main_cmds();
+		else if (len == 6 && !strncmp(spec, "others", 6))
+			list_all_other_cmds();
 		else
 			die(_("unsupported command listing type '%s'"), spec);
 		spec += len;
diff --git a/help.c b/help.c
index bf2738e9ef..71bc001570 100644
--- a/help.c
+++ b/help.c
@@ -297,6 +297,38 @@ void list_common_cmds_help(void)
 	print_cmd_by_category(common_categories);
 }
 
+void list_all_main_cmds(void)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < main_cmds.cnt; i++)
+		puts(main_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
+void list_all_other_cmds(void)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < other_cmds.cnt; i++)
+		puts(other_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index b21d7c94e8..30e165773e 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,8 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_main_cmds(void);
+extern void list_all_other_cmds(void);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 05/10] git.c: convert --list-*builtins to --list-cmds=*

Even if these are hidden options, let's make them a bit more generic since we're introducing more listing types shortly.

This also allows combining multiple listing types, which is usually now (for combining parseopt and builtins) but future types will benefit from this.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  | 27 ++++++++++++++++++++------
 t/t0012-help.sh                        |  2 +-
 3 files changed, 23 insertions(+), 8 deletions(-)
Show changes to 3 files +23 −8

contrib/completion/git-completion.bash, git.c, t/t0012-help.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 01dd9ff07a..d7b448fd94 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -3044,7 +3044,7 @@ __git_complete_common () {
 __git_cmds_with_parseopt_helper=
 __git_support_parseopt_helper () {
 	test -n "$__git_cmds_with_parseopt_helper" ||
-		__git_cmds_with_parseopt_helper="$(__git --list-parseopt-builtins)"
+		__git_cmds_with_parseopt_helper="$(__git --list-cmds=parseopt)"
 
 	case " $__git_cmds_with_parseopt_helper " in
 	*" $1 "*)
diff --git a/git.c b/git.c
index f598fae7b7..ea9bbfb6a3 100644
--- a/git.c
+++ b/git.c
@@ -38,6 +38,25 @@ static int use_pager = -1;
 
 static void list_builtins(unsigned int exclude_option, char sep);
 
+static int list_cmds(const char *spec)
+{
+	while (*spec) {
+		const char *sep = strchrnul(spec, ',');
+		int len = sep - spec;
+
+		if (len == 8 && !strncmp(spec, "builtins", 8))
+			list_builtins(0, '\n');
+		else if (len == 8 && !strncmp(spec, "parseopt", 8))
+			list_builtins(NO_PARSEOPT, ' ');
+		else
+			die(_("unsupported command listing type '%s'"), spec);
+		spec += len;
+		if (*spec == ',')
+			spec++;
+	}
+	return 0;
+}
+
 static void commit_pager_choice(void) {
 	switch (use_pager) {
 	case 0:
@@ -223,12 +242,8 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			}
 			(*argv)++;
 			(*argc)--;
-		} else if (!strcmp(cmd, "--list-builtins")) {
-			list_builtins(0, '\n');
-			exit(0);
-		} else if (!strcmp(cmd, "--list-parseopt-builtins")) {
-			list_builtins(NO_PARSEOPT, ' ');
-			exit(0);
+		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
+			exit(list_cmds(cmd));
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
 			usage(git_usage_string);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index c096f33505..3c91a9024a 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -59,7 +59,7 @@ test_expect_success 'git help' '
 '
 
 test_expect_success 'generate builtin list' '
-	git --list-builtins >builtins
+	git --list-cmds=builtins >builtins
 '
 
 while read builtin
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 10/10] completion: let git provide the completable command list

Instead of maintaining a separate list of command classification, which often could go out of date, let's centralize the information back in git.

While the function in git-completion.bash implies "list porcelain commands", that's not exactly what it does. It gets all commands (aka --list-cmds=main,others) then exclude certain non-porcelain ones. We could almost recreate this list two lists list-mainporcelain and others. The non-porcelain-but-included-anyway is added by the third category list-complete.

list-complete does not recreate exactly the command list before this patch though. The following commands are not part of neither list-mainporcelain nor list-complete and as a result no longer completes:

- annotate             obsolete, discouraged to use
- difftool-helper      not an end user command
- filter-branch        not often used
- get-tar-commit-id    not often used
- imap-send            not often used
- interpreter-trailers not for interactive use
- lost-found           obsolete
- p4                   too short and probably not often used (*)
- peek-remote          deprecated
- svn                  same category as p4 (*)
- tar-tree             obsolete
- verify-commit        not often used

Note that the current completion script incorrectly classifies filter-branch as porcelain and t9902 tests this behavior. We keep it this way in t9902 because this test does not really care which particular command is porcelain or plubmbing, they're just names.

(*) to be fair, send-email command which is in the same foreignscminterface group as svn and p4 does get completion, just because it's used by git and kernel development. So maybe should include them.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 command-list.txt                       |  37 ++++----
 contrib/completion/git-completion.bash | 117 ++++++-------------------
 t/t9902-completion.sh                  |   5 +-
 3 files changed, 48 insertions(+), 111 deletions(-)
Show changes to 3 files +48 −111

command-list.txt, contrib/completion/git-completion.bash, t/t9902-completion.sh

diff --git a/command-list.txt b/command-list.txt
index 99ddc231c1..40776b9587 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -3,11 +3,11 @@
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
-git-apply                               plumbingmanipulators
+git-apply                               plumbingmanipulators            complete
 git-archimport                          foreignscminterface
 git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
-git-blame                               ancillaryinterrogators
+git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
@@ -17,7 +17,7 @@ git-check-mailmap                       purehelpers
 git-checkout                            mainporcelain           history
 git-checkout-index                      plumbingmanipulators
 git-check-ref-format                    purehelpers
-git-cherry                              ancillaryinterrogators
+git-cherry                              ancillaryinterrogators          complete
 git-cherry-pick                         mainporcelain
 git-citool                              mainporcelain
 git-clean                               mainporcelain
@@ -25,7 +25,7 @@ git-clone                               mainporcelain           init
 git-column                              purehelpers
 git-commit                              mainporcelain           history
 git-commit-tree                         plumbingmanipulators
-git-config                              ancillarymanipulators
+git-config                              ancillarymanipulators           complete
 git-count-objects                       ancillaryinterrogators
 git-credential                          purehelpers
 git-credential-cache                    purehelpers
@@ -39,7 +39,7 @@ git-diff                                mainporcelain           history
 git-diff-files                          plumbinginterrogators
 git-diff-index                          plumbinginterrogators
 git-diff-tree                           plumbinginterrogators
-git-difftool                            ancillaryinterrogators
+git-difftool                            ancillaryinterrogators          complete
 git-fast-export                         ancillarymanipulators
 git-fast-import                         ancillarymanipulators
 git-fetch                               mainporcelain           remote
@@ -48,20 +48,20 @@ git-filter-branch                       ancillarymanipulators
 git-fmt-merge-msg                       purehelpers
 git-for-each-ref                        plumbinginterrogators
 git-format-patch                        mainporcelain
-git-fsck                                ancillaryinterrogators
+git-fsck                                ancillaryinterrogators          complete
 git-gc                                  mainporcelain
 git-get-tar-commit-id                   ancillaryinterrogators
 git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
-git-help                                ancillaryinterrogators
+git-help                                ancillaryinterrogators          complete
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
 git-imap-send                           foreignscminterface
 git-index-pack                          plumbingmanipulators
 git-init                                mainporcelain           init
-git-instaweb                            ancillaryinterrogators
+git-instaweb                            ancillaryinterrogators          complete
 git-interpret-trailers                  purehelpers
 gitk                                    mainporcelain
 git-log                                 mainporcelain           info
@@ -75,12 +75,12 @@ git-merge-base                          plumbinginterrogators
 git-merge-file                          plumbingmanipulators
 git-merge-index                         plumbingmanipulators
 git-merge-one-file                      purehelpers
-git-mergetool                           ancillarymanipulators
+git-mergetool                           ancillarymanipulators           complete
 git-merge-tree                          ancillaryinterrogators
 git-mktag                               plumbingmanipulators
 git-mktree                              plumbingmanipulators
 git-mv                                  mainporcelain           worktree
-git-name-rev                            plumbinginterrogators
+git-name-rev                            plumbinginterrogators           complete
 git-notes                               mainporcelain
 git-p4                                  foreignscminterface
 git-pack-objects                        plumbingmanipulators
@@ -96,28 +96,29 @@ git-quiltimport                         foreignscminterface
 git-read-tree                           plumbingmanipulators
 git-rebase                              mainporcelain           history
 git-receive-pack                        synchelpers
-git-reflog                              ancillarymanipulators
-git-remote                              ancillarymanipulators
-git-repack                              ancillarymanipulators
-git-replace                             ancillarymanipulators
-git-request-pull                        foreignscminterface
+git-reflog                              ancillarymanipulators           complete
+git-remote                              ancillarymanipulators           complete
+git-repack                              ancillarymanipulators           complete
+git-replace                             ancillarymanipulators           complete
+git-request-pull                        foreignscminterface             complete
 git-rerere                              ancillaryinterrogators
 git-reset                               mainporcelain           worktree
 git-revert                              mainporcelain
 git-rev-list                            plumbinginterrogators
 git-rev-parse                           ancillaryinterrogators
 git-rm                                  mainporcelain           worktree
-git-send-email                          foreignscminterface
+git-send-email                          foreignscminterface             complete
 git-send-pack                           synchingrepositories
 git-shell                               synchelpers
 git-shortlog                            mainporcelain
 git-show                                mainporcelain           info
-git-show-branch                         ancillaryinterrogators
+git-show-branch                         ancillaryinterrogators          complete
 git-show-index                          plumbinginterrogators
 git-show-ref                            plumbinginterrogators
 git-sh-i18n                             purehelpers
 git-sh-setup                            purehelpers
 git-stash                               mainporcelain
+git-stage                                                               complete
 git-status                              mainporcelain           info
 git-stripspace                          purehelpers
 git-submodule                           mainporcelain
@@ -136,7 +137,7 @@ git-verify-commit                       ancillaryinterrogators
 git-verify-pack                         plumbinginterrogators
 git-verify-tag                          ancillaryinterrogators
 gitweb                                  ancillaryinterrogators
-git-whatchanged                         ancillaryinterrogators
+git-whatchanged                         ancillaryinterrogators          complete
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
 gitattributes                           guide
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 50d14a93dc..b27008b2f0 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -830,18 +830,30 @@ __git_complete_strategy ()
 }
 
 __git_commands () {
-	if test -n "${GIT_TESTING_COMMAND_COMPLETION:-}"
-	then
-		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
-	else
-		git --list-cmds=main,others
-	fi
+	case "$1" in
+	porcelain)
+		if test -n "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		then
+			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		else
+			git --list-cmds=list-mainporcelain,others,list-complete
+		fi
+		;;
+	all)
+		if test -n "$GIT_TESTING_ALL_COMMAND_LIST"
+		then
+			printf "%s" "$GIT_TESTING_ALL_COMMAND_LIST"
+		else
+			git --list-cmds=main,others
+		fi
+		;;
+	esac
 }
 
-__git_list_all_commands ()
+__git_list_commands ()
 {
 	local i IFS=" "$'\n'
-	for i in $(__git_commands)
+	for i in $(__git_commands $1)
 	do
 		case $i in
 		*--*)             : helper pattern;;
@@ -850,6 +862,11 @@ __git_list_all_commands ()
 	done
 }
 
+__git_list_all_commands ()
+{
+	__git_list_commands all
+}
+
 __git_all_commands=
 __git_compute_all_commands ()
 {
@@ -859,89 +876,7 @@ __git_compute_all_commands ()
 
 __git_list_porcelain_commands ()
 {
-	local i IFS=" "$'\n'
-	__git_compute_all_commands
-	for i in $__git_all_commands
-	do
-		case $i in
-		*--*)             : helper pattern;;
-		applymbox)        : ask gittus;;
-		applypatch)       : ask gittus;;
-		archimport)       : import;;
-		cat-file)         : plumbing;;
-		check-attr)       : plumbing;;
-		check-ignore)     : plumbing;;
-		check-mailmap)    : plumbing;;
-		check-ref-format) : plumbing;;
-		checkout-index)   : plumbing;;
-		column)           : internal helper;;
-		commit-tree)      : plumbing;;
-		count-objects)    : infrequent;;
-		credential)       : credentials;;
-		credential-*)     : credentials helper;;
-		cvsexportcommit)  : export;;
-		cvsimport)        : import;;
-		cvsserver)        : daemon;;
-		daemon)           : daemon;;
-		diff-files)       : plumbing;;
-		diff-index)       : plumbing;;
-		diff-tree)        : plumbing;;
-		fast-import)      : import;;
-		fast-export)      : export;;
-		fsck-objects)     : plumbing;;
-		fetch-pack)       : plumbing;;
-		fmt-merge-msg)    : plumbing;;
-		for-each-ref)     : plumbing;;
-		hash-object)      : plumbing;;
-		http-*)           : transport;;
-		index-pack)       : plumbing;;
-		init-db)          : deprecated;;
-		local-fetch)      : plumbing;;
-		ls-files)         : plumbing;;
-		ls-remote)        : plumbing;;
-		ls-tree)          : plumbing;;
-		mailinfo)         : plumbing;;
-		mailsplit)        : plumbing;;
-		merge-*)          : plumbing;;
-		mktree)           : plumbing;;
-		mktag)            : plumbing;;
-		pack-objects)     : plumbing;;
-		pack-redundant)   : plumbing;;
-		pack-refs)        : plumbing;;
-		parse-remote)     : plumbing;;
-		patch-id)         : plumbing;;
-		prune)            : plumbing;;
-		prune-packed)     : plumbing;;
-		quiltimport)      : import;;
-		read-tree)        : plumbing;;
-		receive-pack)     : plumbing;;
-		remote-*)         : transport;;
-		rerere)           : plumbing;;
-		rev-list)         : plumbing;;
-		rev-parse)        : plumbing;;
-		runstatus)        : plumbing;;
-		sh-setup)         : internal;;
-		shell)            : daemon;;
-		show-ref)         : plumbing;;
-		send-pack)        : plumbing;;
-		show-index)       : plumbing;;
-		ssh-*)            : transport;;
-		stripspace)       : plumbing;;
-		symbolic-ref)     : plumbing;;
-		unpack-file)      : plumbing;;
-		unpack-objects)   : plumbing;;
-		update-index)     : plumbing;;
-		update-ref)       : plumbing;;
-		update-server-info) : daemon;;
-		upload-archive)   : plumbing;;
-		upload-pack)      : plumbing;;
-		write-tree)       : plumbing;;
-		var)              : infrequent;;
-		verify-pack)      : infrequent;;
-		verify-tag)       : plumbing;;
-		*) echo $i;;
-		esac
-	done
+	__git_list_commands porcelain
 }
 
 __git_porcelain_commands=
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 1b34caa1e1..2f16679380 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -13,7 +13,7 @@ complete ()
 	return 0
 }
 
-# Be careful when updating this list:
+# Be careful when updating these lists:
 #
 # (1) The build tree may have build artifact from different branch, or
 #     the user's $PATH may have a random executable that may begin
@@ -30,7 +30,8 @@ complete ()
 #     completion for "git <TAB>", and a plumbing is excluded.  "add",
 #     "filter-branch" and "ls-files" are listed for this.
 
-GIT_TESTING_COMMAND_COMPLETION='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_ALL_COMMAND_LIST='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_PORCELAIN_COMMAND_LIST='add checkout filter-branch'
 
 . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash"
 
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 04/10] Remove common-cmds.h

After the last patch, common-cmds.h is no longer used (and it was actually broken). Remove all related code. command-list.h will take its place from now on.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 .gitignore          |  1 -
 Makefile            | 17 ++++++-----------
 generate-cmdlist.sh | 46 +++------------------------------------------
 3 files changed, 9 insertions(+), 55 deletions(-)
Show changes to 3 files +9 −55

.gitignore, Makefile, generate-cmdlist.sh

diff --git a/.gitignore b/.gitignore
index d4c3914167..0836083992 100644
--- a/.gitignore
+++ b/.gitignore
@@ -179,7 +179,6 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
-/common-cmds.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index ab67150e68..71b5b594cd 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h command-list.h
+GENERATED_H += command-list.h
 
 LIB_H = $(shell $(FIND) . \
 	-name .git -prune -o \
@@ -1916,9 +1916,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
-help.sp help.s help.o: common-cmds.h command-list.h
+help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -1937,11 +1937,6 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
-common-cmds.h: generate-cmdlist.sh command-list.txt
-
-common-cmds.h: $(wildcard Documentation/git-*.txt)
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON >$@+ && mv $@+ $@
-
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git-*.txt)
@@ -2155,7 +2150,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h or command-list.h
+# Dependencies on automatically generated headers such as command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2534,7 +2529,7 @@ sparse: $(SP_OBJ)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h command-list.h
+check: command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -2782,7 +2777,7 @@ clean: profile-clean coverage-clean
 	$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 93de8e8f59..015eef2804 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -68,46 +68,6 @@ struct cmdname_help {
 	uint32_t category;
 };
 "
-if [ -z "$2" ]
-then
-	define_categories "$1"
-	echo
-	print_command_list "$1"
-	exit 0
-fi
-
-echo "static const char *common_cmd_groups[] = {"
-
-grps=grps$$.tmp
-match=match$$.tmp
-trap "rm -f '$grps' '$match'" 0 1 2 3 15
-
-sed -n '
-	1,/^### common groups/b
-	/^### command list/q
-	/^#/b
-	/^[ 	]*$/b
-	h;s/^[^ 	][^ 	]*[ 	][ 	]*\(.*\)/	N_("\1"),/p
-	g;s/^\([^ 	][^ 	]*\)[ 	].*/\1/w '$grps'
-	' "$1"
-printf '};\n\n'
-
-n=0
-substnum=
-while read grp
-do
-	echo "^git-..*[ 	]$grp"
-	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
-	n=$(($n+1))
-done <"$grps" >"$match"
-
-printf 'static struct cmdname_help common_cmds[] = {\n'
-grep -f "$match" "$1" |
-sed 's/^git-//' |
-sort |
-while read cmd tags
-do
-	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
-	echo "	{\"$cmd\", $(get_synopsis git-$cmd), $tag},"
-done
-echo "};"
+define_categories "$1"
+echo
+print_command_list "$1"
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 07/10] git: support --list-cmds=list-<category>

This allows us to select any group of commands by a category defined in command-list.txt. This is an internal/hidden option so we don't have to be picky about the category name or worried about exposing too much.

This will be used later by git-completion.bash to retrieve certain command groups.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 generate-cmdlist.sh | 17 +++++++++++++++++
 git.c               |  7 +++++++
 help.c              | 22 ++++++++++++++++++++++
 help.h              |  1 +
 4 files changed, 47 insertions(+)
Show changes to 4 files +47 −0

generate-cmdlist.sh, git.c, help.c, help.h

diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 015eef2804..bfd8ef0671 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -45,6 +45,21 @@ define_categories() {
 	test "$bit" -gt 32 && die "Urgh.. too many categories?"
 }
 
+define_category_names() {
+	echo
+	echo "/* Category names */"
+	echo "static const char *category_names[] = {"
+	bit=0
+	category_list "$1" |
+	while read cat
+	do
+		echo "	\"$cat\", /* (1UL << $bit) */"
+		bit=$(($bit+1))
+	done
+	echo "	NULL"
+	echo "};"
+}
+
 print_command_list() {
 	echo "static struct cmdname_help command_list[] = {"
 
@@ -70,4 +85,6 @@ struct cmdname_help {
 "
 define_categories "$1"
 echo
+define_category_names "$1"
+echo
 print_command_list "$1"
diff --git a/git.c b/git.c
index da161a74ae..93927e34c4 100644
--- a/git.c
+++ b/git.c
@@ -52,6 +52,13 @@ static int list_cmds(const char *spec)
 			list_all_main_cmds();
 		else if (len == 6 && !strncmp(spec, "others", 6))
 			list_all_other_cmds();
+		else if (len > 5 && !strncmp(spec, "list-", 5)) {
+			struct strbuf sb = STRBUF_INIT;
+
+			strbuf_add(&sb, spec + 5, len - 5);
+			list_cmds_by_category(sb.buf);
+			strbuf_release(&sb);
+		}
 		else
 			die(_("unsupported command listing type '%s'"), spec);
 		spec += len;
diff --git a/help.c b/help.c
index 71bc001570..156dde1bea 100644
--- a/help.c
+++ b/help.c
@@ -329,6 +329,28 @@ void list_all_other_cmds(void)
 	clean_cmdnames(&other_cmds);
 }
 
+void list_cmds_by_category(const char *cat)
+{
+	int i, n = ARRAY_SIZE(command_list);
+	uint32_t cat_id = 0;
+
+	for (i = 0; category_names[i]; i++) {
+		if (!strcmp(cat, category_names[i])) {
+			cat_id = 1UL << i;
+			break;
+		}
+	}
+	if (!cat_id)
+		die(_("unsupported command listing type '%s'"), cat);
+
+	for (i = 0; i < n; i++) {
+		struct cmdname_help *cmd = command_list + i;
+
+		if (cmd->category & cat_id)
+			puts(drop_prefix(cmd->name));
+	}
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 30e165773e..752368840a 100644
--- a/help.h
+++ b/help.h
@@ -19,6 +19,7 @@ static inline void mput_char(char c, unsigned int num)
 extern void list_common_cmds_help(void);
 extern void list_all_main_cmds(void);
 extern void list_all_other_cmds(void);
+extern void list_cmds_by_category(const char *category);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 08/10] help: add "-a --verbose" to list all commands with synopsis

This lists all recognized commands [1] by category. The group order follows closely git.txt.

[1] We may actually show commands that are not built (e.g. if you set NO_PERL you don't have git-instaweb but it's still listed here). I ignore the problem because on Linux a git package could be split anyway. The "git-core" package may not contain git-instaweb even if it's built because it may end up in a separate package. We can't know anyway.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-help.txt |  4 +++-
 builtin/help.c             |  7 +++++++
 help.c                     | 16 ++++++++++++++++
 help.h                     |  1 +
 t/t0012-help.sh            |  9 +++++++++
 5 files changed, 36 insertions(+), 1 deletion(-)
Show changes to 5 files +36 −1

Documentation/git-help.txt, builtin/help.c, help.c, help.h, t/t0012-help.sh

diff --git a/Documentation/git-help.txt b/Documentation/git-help.txt
index 40d328a4b3..a40fc38d8b 100644
--- a/Documentation/git-help.txt
+++ b/Documentation/git-help.txt
@@ -8,7 +8,7 @@ git-help - Display help information about Git
 SYNOPSIS
 --------
 [verse]
-'git help' [-a|--all] [-g|--guide]
+'git help' [-a|--all [--verbose]] [-g|--guide]
 	   [-i|--info|-m|--man|-w|--web] [COMMAND|GUIDE]
 
 DESCRIPTION
@@ -42,6 +42,8 @@ OPTIONS
 --all::
 	Prints all the available commands on the standard output. This
 	option overrides any given command or guide name.
+	When used with `--verbose` print description for all recognized
+	commands.
 
 -g::
 --guides::
diff --git a/builtin/help.c b/builtin/help.c
index 2d51071429..83a7d73afe 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -36,6 +36,7 @@ static const char *html_path;
 
 static int show_all = 0;
 static int show_guides = 0;
+static int verbose;
 static unsigned int colopts;
 static enum help_format help_format = HELP_FORMAT_NONE;
 static int exclude_guides;
@@ -48,6 +49,7 @@ static struct option builtin_help_options[] = {
 			HELP_FORMAT_WEB),
 	OPT_SET_INT('i', "info", &help_format, N_("show info page"),
 			HELP_FORMAT_INFO),
+	OPT__VERBOSE(&verbose, N_("print command description")),
 	OPT_END(),
 };
 
@@ -463,6 +465,11 @@ int cmd_help(int argc, const char **argv, const char *prefix)
 
 	if (show_all) {
 		git_config(git_help_config, NULL);
+		if (verbose) {
+			setup_pager();
+			list_all_cmds_help();
+			return 0;
+		}
 		printf(_("usage: %s%s"), _(git_usage_string), "\n\n");
 		load_command_list("git-", &main_cmds, &other_cmds);
 		list_commands(colopts, &main_cmds, &other_cmds);
diff --git a/help.c b/help.c
index 156dde1bea..f9da0214f1 100644
--- a/help.c
+++ b/help.c
@@ -27,6 +27,17 @@ static struct category_description common_categories[] = {
 	{ CAT_remote, N_("collaborate (see also: git help workflows)") },
 	{ 0, NULL }
 };
+static struct category_description main_categories[] = {
+	{ CAT_mainporcelain, N_("Main Porcelain Commands") },
+	{ CAT_ancillarymanipulators, N_("Ancillary Commands / Manipulators") },
+	{ CAT_ancillaryinterrogators, N_("Ancillary Commands / Interrogators") },
+	{ CAT_foreignscminterface, N_("Interacting with Others") },
+	{ CAT_plumbingmanipulators, N_("Low-level Commands / Manipulators") },
+	{ CAT_plumbinginterrogators, N_("Low-level Commands / Interrogators") },
+	{ CAT_synchingrepositories, N_("Low-level Commands / Synching Repositories") },
+	{ CAT_purehelpers, N_("Low-level Commands / Internal Helpers") },
+	{ 0, NULL }
+};
 
 static const char *drop_prefix(const char *name)
 {
@@ -351,6 +362,11 @@ void list_cmds_by_category(const char *cat)
 	}
 }
 
+void list_all_cmds_help(void)
+{
+	print_cmd_by_category(main_categories);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 752368840a..090d46ba01 100644
--- a/help.h
+++ b/help.h
@@ -17,6 +17,7 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds_help(void);
 extern void list_all_main_cmds(void);
 extern void list_all_other_cmds(void);
 extern void list_cmds_by_category(const char *category);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 3c91a9024a..060df24c2d 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -25,6 +25,15 @@ test_expect_success "setup" '
 	EOF
 '
 
+# make sure to exercise these code paths, the output is a bit tricky
+# to verify
+test_expect_success 'basic help commands' '
+	git help >/dev/null &&
+	git help -a >/dev/null &&
+	git help -g >/dev/null &&
+	git help -av >/dev/null
+'
+
 test_expect_success "works for commands and guides by default" '
 	configure_help &&
 	git help status &&
-- 
2.17.0.664.g8924eee37a
Nguyễn Thái Ngọc Duy· Apr 29, 2018, 18:18 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v5 09/10] help: use command-list.txt for the source of guides

The help command currently hard codes the list of guides and their summary in C. Let's move this list to command-list.txt. This lets us extract summary lines from Documentation/git*.txt. This also potentially lets us list guides in git.txt, but I'll leave that for now.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/gitattributes.txt        |  2 +-
 Documentation/gitmodules.txt           |  2 +-
 Documentation/gitrevisions.txt         |  2 +-
 Makefile                               |  2 +-
 builtin/help.c                         | 32 --------------------------
 command-list.txt                       | 16 +++++++++++++
 contrib/completion/git-completion.bash | 15 ++++++++----
 help.c                                 | 18 ++++++++++++---
 help.h                                 |  1 +
 t/t0012-help.sh                        |  6 +++++
 10 files changed, 52 insertions(+), 44 deletions(-)
Show changes to 10 files +52 −44

Documentation/gitattributes.txt, Documentation/gitmodules.txt, Documentation/gitrevisions.txt, Makefile, builtin/help.c, command-list.txt, contrib/completion/git-completion.bash, help.c, help.h, t/t0012-help.sh

diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 1094fe2b5b..083c2f380d 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -3,7 +3,7 @@ gitattributes(5)
 
 NAME
 ----
-gitattributes - defining attributes per path
+gitattributes - Defining attributes per path
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index db5d47eb19..4d63def206 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -3,7 +3,7 @@ gitmodules(5)
 
 NAME
 ----
-gitmodules - defining submodule properties
+gitmodules - Defining submodule properties
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitrevisions.txt b/Documentation/gitrevisions.txt
index 27dec5b91d..1f6cceaefb 100644
--- a/Documentation/gitrevisions.txt
+++ b/Documentation/gitrevisions.txt
@@ -3,7 +3,7 @@ gitrevisions(7)
 
 NAME
 ----
-gitrevisions - specifying revisions and ranges for Git
+gitrevisions - Specifying revisions and ranges for Git
 
 SYNOPSIS
 --------
diff --git a/Makefile b/Makefile
index 71b5b594cd..18696e35b0 100644
--- a/Makefile
+++ b/Makefile
@@ -1939,7 +1939,7 @@ $(BUILT_INS): git$X
 
 command-list.h: generate-cmdlist.sh command-list.txt
 
-command-list.h: $(wildcard Documentation/git-*.txt)
+command-list.h: $(wildcard Documentation/git*.txt)
 	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
diff --git a/builtin/help.c b/builtin/help.c
index 83a7d73afe..b58e8d5f6a 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -402,38 +402,6 @@ static void show_html_page(const char *git_cmd)
 	open_html(page_path.buf);
 }
 
-static struct {
-	const char *name;
-	const char *help;
-} common_guides[] = {
-	{ "attributes", N_("Defining attributes per path") },
-	{ "everyday", N_("Everyday Git With 20 Commands Or So") },
-	{ "glossary", N_("A Git glossary") },
-	{ "ignore", N_("Specifies intentionally untracked files to ignore") },
-	{ "modules", N_("Defining submodule properties") },
-	{ "revisions", N_("Specifying revisions and ranges for Git") },
-	{ "tutorial", N_("A tutorial introduction to Git (for version 1.5.1 or newer)") },
-	{ "workflows", N_("An overview of recommended workflows with Git") },
-};
-
-static void list_common_guides_help(void)
-{
-	int i, longest = 0;
-
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		if (longest < strlen(common_guides[i].name))
-			longest = strlen(common_guides[i].name);
-	}
-
-	puts(_("The common Git guides are:\n"));
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		printf("   %s   ", common_guides[i].name);
-		mput_char(' ', longest - strlen(common_guides[i].name));
-		puts(_(common_guides[i].help));
-	}
-	putchar('\n');
-}
-
 static const char *check_git_cmd(const char* cmd)
 {
 	char *alias;
diff --git a/command-list.txt b/command-list.txt
index 3bd23201a6..99ddc231c1 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -139,3 +139,19 @@ gitweb                                  ancillaryinterrogators
 git-whatchanged                         ancillaryinterrogators
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
+gitattributes                           guide
+gitcli                                  guide
+gitcore-tutorial                        guide
+gitcvs-migration                        guide
+gitdiffcore                             guide
+giteveryday                             guide
+gitglossary                             guide
+githooks                                guide
+gitignore                               guide
+gitmodules                              guide
+gitnamespaces                           guide
+gitrepository-layout                    guide
+gitrevisions                            guide
+gittutorial-2                           guide
+gittutorial                             guide
+gitworkflows                            guide
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 77cfb8a20b..50d14a93dc 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1570,6 +1570,13 @@ _git_grep ()
 	__git_complete_refs
 }
 
+__git_all_guides=
+__git_compute_all_guides ()
+{
+	test -n "$__git_all_guides" ||
+	__git_all_guides=$(git --list-cmds=list-guide)
+}
+
 _git_help ()
 {
 	case "$cur" in
@@ -1579,11 +1586,9 @@ _git_help ()
 		;;
 	esac
 	__git_compute_all_commands
-	__gitcomp "$__git_all_commands $(__git_aliases)
-		attributes cli core-tutorial cvs-migration
-		diffcore everyday gitk glossary hooks ignore modules
-		namespaces repository-layout revisions tutorial tutorial-2
-		workflows
+	__git_compute_all_guides
+	__gitcomp "$__git_all_commands $(__git_aliases) $__git_all_guides
+		gitk
 		"
 }
 
diff --git a/help.c b/help.c
index f9da0214f1..f4629d0ee1 100644
--- a/help.c
+++ b/help.c
@@ -39,12 +39,14 @@ static struct category_description main_categories[] = {
 	{ 0, NULL }
 };
 
-static const char *drop_prefix(const char *name)
+static const char *drop_prefix(const char *name, uint32_t category)
 {
 	const char *new_name;
 
 	if (skip_prefix(name, "git-", &new_name))
 		return new_name;
+	if (category == CAT_guide && skip_prefix(name, "git", &new_name))
+		return new_name;
 	return name;
 
 }
@@ -66,7 +68,7 @@ static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
 			continue;
 
 		cmds[nr] = *cmd;
-		cmds[nr].name = drop_prefix(cmd->name);
+		cmds[nr].name = drop_prefix(cmd->name, cmd->category);
 
 		nr++;
 	}
@@ -358,10 +360,20 @@ void list_cmds_by_category(const char *cat)
 		struct cmdname_help *cmd = command_list + i;
 
 		if (cmd->category & cat_id)
-			puts(drop_prefix(cmd->name));
+			puts(drop_prefix(cmd->name, cmd->category));
 	}
 }
 
+void list_common_guides_help(void)
+{
+	struct category_description catdesc[] = {
+		{ CAT_guide, N_("The common Git guides are:") },
+		{ 0, NULL }
+	};
+	print_cmd_by_category(catdesc);
+	putchar('\n');
+}
+
 void list_all_cmds_help(void)
 {
 	print_cmd_by_category(main_categories);
diff --git a/help.h b/help.h
index 090d46ba01..5d27368fe4 100644
--- a/help.h
+++ b/help.h
@@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds_help(void);
+extern void list_common_guides_help(void);
 extern void list_all_main_cmds(void);
 extern void list_all_other_cmds(void);
 extern void list_cmds_by_category(const char *category);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 060df24c2d..bc27df7f38 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -66,6 +66,12 @@ test_expect_success 'git help' '
 	test_i18ngrep "^   commit " help.output &&
 	test_i18ngrep "^   fetch  " help.output
 '
+test_expect_success 'git help -g' '
+	git help -g >help.output &&
+	test_i18ngrep "^   attributes " help.output &&
+	test_i18ngrep "^   everyday   " help.output &&
+	test_i18ngrep "^   tutorial   " help.output
+'
 
 test_expect_success 'generate builtin list' '
 	git --list-cmds=builtins >builtins
-- 
2.17.0.664.g8924eee37a
Duy Nguyen· Apr 29, 2018, 18:23 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v5 09/10] help: use command-list.txt for the source of guides

Phillip (and others) the changes in this patch make "git help -g" now lists a lot more guides than just the "common" one as advertised (see below for the exact list). The man page for "git help -g" also mentions that it would list "useful" guides, not all guides. But we have no way to list all guides as far as I can tell.

I guess we have two options forward:
- keep "help -g" to common guide (we can tag common guides in
command-list.txt) and add a new option to list all guides ("help
-ag"?)
- reword the man page to make "help -g" list all guides
I'm ok with either direction. What's your preference?
For comparison, this is the new output
The common Git guides are:
   attributes          Defining attributes per path
   cli                 Git command-line interface and conventions
   core-tutorial       A Git core tutorial for developers
   cvs-migration       Git for CVS users
   diffcore            Tweaking diff output
   everyday            A useful minimum set of commands for Everyday
Git
   glossary            A Git Glossary
   hooks               Hooks used by Git
   ignore              Specifies intentionally untracked files to
ignore
   modules             Defining submodule properties
   namespaces          Git namespaces
   repository-layout   Git Repository Layout
   revisions           Specifying revisions and ranges for Git
   tutorial            A tutorial introduction to Git
   tutorial-2          A tutorial introduction to Git: part two
   workflows           An overview of recommended workflows with Git
compared to the old version
The common Git guides are:
   attributes   Defining attributes per path
   everyday     Everyday Git With 20 Commands Or So
   glossary     A Git glossary
   ignore       Specifies intentionally untracked files to ignore
   modules      Defining submodule properties
   revisions    Specifying revisions and ranges for Git
   tutorial     A tutorial introduction to Git (for version 1.5.1 or
newer)
   workflows    An overview of recommended workflows with Git
On Sun, Apr 29, 2018 at 8:18 PM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:
Show 255 quoted lines
> The help command currently hard codes the list of guides and their
> summary in C. Let's move this list to command-list.txt. This lets us
> extract summary lines from Documentation/git*.txt. This also
> potentially lets us list guides in git.txt, but I'll leave that for
> now.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  Documentation/gitattributes.txt        |  2 +-
>  Documentation/gitmodules.txt           |  2 +-
>  Documentation/gitrevisions.txt         |  2 +-
>  Makefile                               |  2 +-
>  builtin/help.c                         | 32 --------------------------
>  command-list.txt                       | 16 +++++++++++++
>  contrib/completion/git-completion.bash | 15 ++++++++----
>  help.c                                 | 18 ++++++++++++---
>  help.h                                 |  1 +
>  t/t0012-help.sh                        |  6 +++++
>  10 files changed, 52 insertions(+), 44 deletions(-)
>
> diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
> index 1094fe2b5b..083c2f380d 100644
> --- a/Documentation/gitattributes.txt
> +++ b/Documentation/gitattributes.txt
> @@ -3,7 +3,7 @@ gitattributes(5)
>
>  NAME
>  ----
> -gitattributes - defining attributes per path
> +gitattributes - Defining attributes per path
>
>  SYNOPSIS
>  --------
> diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
> index db5d47eb19..4d63def206 100644
> --- a/Documentation/gitmodules.txt
> +++ b/Documentation/gitmodules.txt
> @@ -3,7 +3,7 @@ gitmodules(5)
>
>  NAME
>  ----
> -gitmodules - defining submodule properties
> +gitmodules - Defining submodule properties
>
>  SYNOPSIS
>  --------
> diff --git a/Documentation/gitrevisions.txt b/Documentation/gitrevisions.txt
> index 27dec5b91d..1f6cceaefb 100644
> --- a/Documentation/gitrevisions.txt
> +++ b/Documentation/gitrevisions.txt
> @@ -3,7 +3,7 @@ gitrevisions(7)
>
>  NAME
>  ----
> -gitrevisions - specifying revisions and ranges for Git
> +gitrevisions - Specifying revisions and ranges for Git
>
>  SYNOPSIS
>  --------
> diff --git a/Makefile b/Makefile
> index 71b5b594cd..18696e35b0 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1939,7 +1939,7 @@ $(BUILT_INS): git$X
>
>  command-list.h: generate-cmdlist.sh command-list.txt
>
> -command-list.h: $(wildcard Documentation/git-*.txt)
> +command-list.h: $(wildcard Documentation/git*.txt)
>         $(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
>
>  SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
> diff --git a/builtin/help.c b/builtin/help.c
> index 83a7d73afe..b58e8d5f6a 100644
> --- a/builtin/help.c
> +++ b/builtin/help.c
> @@ -402,38 +402,6 @@ static void show_html_page(const char *git_cmd)
>         open_html(page_path.buf);
>  }
>
> -static struct {
> -       const char *name;
> -       const char *help;
> -} common_guides[] = {
> -       { "attributes", N_("Defining attributes per path") },
> -       { "everyday", N_("Everyday Git With 20 Commands Or So") },
> -       { "glossary", N_("A Git glossary") },
> -       { "ignore", N_("Specifies intentionally untracked files to ignore") },
> -       { "modules", N_("Defining submodule properties") },
> -       { "revisions", N_("Specifying revisions and ranges for Git") },
> -       { "tutorial", N_("A tutorial introduction to Git (for version 1.5.1 or newer)") },
> -       { "workflows", N_("An overview of recommended workflows with Git") },
> -};
> -
> -static void list_common_guides_help(void)
> -{
> -       int i, longest = 0;
> -
> -       for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
> -               if (longest < strlen(common_guides[i].name))
> -                       longest = strlen(common_guides[i].name);
> -       }
> -
> -       puts(_("The common Git guides are:\n"));
> -       for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
> -               printf("   %s   ", common_guides[i].name);
> -               mput_char(' ', longest - strlen(common_guides[i].name));
> -               puts(_(common_guides[i].help));
> -       }
> -       putchar('\n');
> -}
> -
>  static const char *check_git_cmd(const char* cmd)
>  {
>         char *alias;
> diff --git a/command-list.txt b/command-list.txt
> index 3bd23201a6..99ddc231c1 100644
> --- a/command-list.txt
> +++ b/command-list.txt
> @@ -139,3 +139,19 @@ gitweb                                  ancillaryinterrogators
>  git-whatchanged                         ancillaryinterrogators
>  git-worktree                            mainporcelain
>  git-write-tree                          plumbingmanipulators
> +gitattributes                           guide
> +gitcli                                  guide
> +gitcore-tutorial                        guide
> +gitcvs-migration                        guide
> +gitdiffcore                             guide
> +giteveryday                             guide
> +gitglossary                             guide
> +githooks                                guide
> +gitignore                               guide
> +gitmodules                              guide
> +gitnamespaces                           guide
> +gitrepository-layout                    guide
> +gitrevisions                            guide
> +gittutorial-2                           guide
> +gittutorial                             guide
> +gitworkflows                            guide
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index 77cfb8a20b..50d14a93dc 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -1570,6 +1570,13 @@ _git_grep ()
>         __git_complete_refs
>  }
>
> +__git_all_guides=
> +__git_compute_all_guides ()
> +{
> +       test -n "$__git_all_guides" ||
> +       __git_all_guides=$(git --list-cmds=list-guide)
> +}
> +
>  _git_help ()
>  {
>         case "$cur" in
> @@ -1579,11 +1586,9 @@ _git_help ()
>                 ;;
>         esac
>         __git_compute_all_commands
> -       __gitcomp "$__git_all_commands $(__git_aliases)
> -               attributes cli core-tutorial cvs-migration
> -               diffcore everyday gitk glossary hooks ignore modules
> -               namespaces repository-layout revisions tutorial tutorial-2
> -               workflows
> +       __git_compute_all_guides
> +       __gitcomp "$__git_all_commands $(__git_aliases) $__git_all_guides
> +               gitk
>                 "
>  }
>
> diff --git a/help.c b/help.c
> index f9da0214f1..f4629d0ee1 100644
> --- a/help.c
> +++ b/help.c
> @@ -39,12 +39,14 @@ static struct category_description main_categories[] = {
>         { 0, NULL }
>  };
>
> -static const char *drop_prefix(const char *name)
> +static const char *drop_prefix(const char *name, uint32_t category)
>  {
>         const char *new_name;
>
>         if (skip_prefix(name, "git-", &new_name))
>                 return new_name;
> +       if (category == CAT_guide && skip_prefix(name, "git", &new_name))
> +               return new_name;
>         return name;
>
>  }
> @@ -66,7 +68,7 @@ static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
>                         continue;
>
>                 cmds[nr] = *cmd;
> -               cmds[nr].name = drop_prefix(cmd->name);
> +               cmds[nr].name = drop_prefix(cmd->name, cmd->category);
>
>                 nr++;
>         }
> @@ -358,10 +360,20 @@ void list_cmds_by_category(const char *cat)
>                 struct cmdname_help *cmd = command_list + i;
>
>                 if (cmd->category & cat_id)
> -                       puts(drop_prefix(cmd->name));
> +                       puts(drop_prefix(cmd->name, cmd->category));
>         }
>  }
>
> +void list_common_guides_help(void)
> +{
> +       struct category_description catdesc[] = {
> +               { CAT_guide, N_("The common Git guides are:") },
> +               { 0, NULL }
> +       };
> +       print_cmd_by_category(catdesc);
> +       putchar('\n');
> +}
> +
>  void list_all_cmds_help(void)
>  {
>         print_cmd_by_category(main_categories);
> diff --git a/help.h b/help.h
> index 090d46ba01..5d27368fe4 100644
> --- a/help.h
> +++ b/help.h
> @@ -18,6 +18,7 @@ static inline void mput_char(char c, unsigned int num)
>
>  extern void list_common_cmds_help(void);
>  extern void list_all_cmds_help(void);
> +extern void list_common_guides_help(void);
>  extern void list_all_main_cmds(void);
>  extern void list_all_other_cmds(void);
>  extern void list_cmds_by_category(const char *category);
> diff --git a/t/t0012-help.sh b/t/t0012-help.sh
> index 060df24c2d..bc27df7f38 100755
> --- a/t/t0012-help.sh
> +++ b/t/t0012-help.sh
> @@ -66,6 +66,12 @@ test_expect_success 'git help' '
>         test_i18ngrep "^   commit " help.output &&
>         test_i18ngrep "^   fetch  " help.output
>  '
> +test_expect_success 'git help -g' '
> +       git help -g >help.output &&
> +       test_i18ngrep "^   attributes " help.output &&
> +       test_i18ngrep "^   everyday   " help.output &&
> +       test_i18ngrep "^   tutorial   " help.output
> +'
>
>  test_expect_success 'generate builtin list' '
>         git --list-cmds=builtins >builtins
> --
> 2.17.0.664.g8924eee37a
>
-- 
Duy
Duy Nguyen· Apr 30, 2018, 15:53 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v5 00/10] Keep all info in command-list.txt in git binary

This is probably scope creep for this series, but do you guys think we should do the same for config variables completion? We currently maintain a giant list at the end of _git_config(). Extracting the list from Documentation/config.txt to keep it in a C array does not look super hard. There will be some special handling for advice.* or color.<slot>... but overall I still think it's a net gain.

Listing all recognizable config variables from "git config" (or "git help") would be lovely, but I don't think it helps unless we could print a short summary line each variable, but this info is not available anywhere. -- Duy

Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 00/13] Keep all info in command-list.txt in git binary

v6 is now "feature complete". v6 adds
- documentation in command-list.txt so people know how to update it
- support for config key completion.commands, which consists
  of extra commands. It could also be used for excluding some
  commands.
Interdiff
Show changes to 5 files +156 −20

command-list.txt, contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/command-list.txt b/command-list.txt
index 40776b9587..3e21ddfcfb 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -1,3 +1,47 @@
+# Command classification list
+# ---------------------------
+# All supported commands, builtin or external, must be described in
+# here. This info is used to list commands in various places. Each
+# command is on one line followed by one or more attributes.
+#
+# The first attribute group is mandatory and indicates the command
+# type. This group includes:
+#
+#   mainporcelain
+#   ancillarymanipulators
+#   ancillaryinterrogators
+#   foreignscminterface
+#   plumbingmanipulators
+#   plumbinginterrogators
+#   synchingrepositories
+#   synchelpers
+#   purehelpers
+#
+# The type names are self explanatory. But if you want to see what
+# command belongs to what group to get a better picture, have a look
+# at "git" man page, "GIT COMMANDS" section.
+#
+# Commands of type mainporcelain can also optionally have one of these
+# attributes:
+#
+#   init
+#   worktree
+#   info
+#   history
+#   remote
+#
+# These commands are considered "common" and will show up in "git
+# help" output in groups. Uncommon porcelain commands must not
+# specify any of these attributes.
+#
+# "complete" attribute is used to mark that the command should be
+# completable by git-completion.bash. Note that by default,
+# mainporcelain commands are completable so you don't need this
+# attribute.
+#
+# While not true commands, guides are also specified here, which can
+# only have "guide" attribute and nothing else.
+#
 ### command list (do not change this line, also do not change alignment)
 # command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 908692ea52..0fd29803d5 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -38,6 +38,38 @@
 #
 #     When set to "1", do not include "DWIM" suggestions in git-checkout
 #     completion (e.g., completing "foo" when "origin/foo" exists).
+#
+#   GIT_COMPLETION_CMD_GROUPS
+#
+#     When set, "git --list-cmds=$GIT_COMPLETION_CMD_GROUPS" will be
+#     used to get the list of completable commands. The default is
+#     "mainporcelain,others,list-complete" (in English: all porcelain
+#     commands and external ones are included. Certain non-porcelain
+#     commands are also marked for completion in command-list.txt).
+#     You could for example complete all commands with
+#
+#         GIT_COMPLETION_CMD_GROUPS=main,others
+#
+#     Or you could go with defaults add some extra commands specified
+#     in the configuration variable completion.commands [1] with
+#
+#         GIT_COMPLETION_CMD_GROUPS=mainporcelain,others,list-complete,config
+#
+#     Or go completely custom group with
+#
+#         GIT_COMPLETION_CMD_GROUPS=config
+#
+#     Or you could even play with other command categories found in
+#     command-list.txt.
+#
+#     [1] Note that completion.commands should not be per-repository
+#         since the command list is generated once and cached.
+#
+#         completion.commands could be used to exclude commands as
+#         well.  If a command in this list begins with '-', then it
+#         will be excluded from the list of commands gathered by the
+#         groups specified before "config" in
+#         $GIT_COMPLETION_CMD_GROUPS.
 
 case "$COMP_WORDBREAKS" in
 *:*) : great ;;
@@ -840,6 +872,9 @@ __git_commands () {
 		if test -n "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
 		then
 			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		elif test -n "$GIT_COMPLETION_CMD_GROUPS"
+		then
+			git --list-cmds="$GIT_COMPLETION_CMD_GROUPS"
 		else
 			git --list-cmds=list-mainporcelain,others,list-complete
 		fi
diff --git a/git.c b/git.c
index 1c8b0c93e1..fd08911e11 100644
--- a/git.c
+++ b/git.c
@@ -36,27 +36,30 @@ const char git_more_info_string[] =
 
 static int use_pager = -1;
 
-static void list_builtins(unsigned int exclude_option, char sep);
+static void list_builtins(struct string_list *list, unsigned int exclude_option);
 
 static int list_cmds(const char *spec)
 {
+	struct string_list list = STRING_LIST_INIT_DUP;
+	int i;
+
 	while (*spec) {
 		const char *sep = strchrnul(spec, ',');
 		int len = sep - spec;
 
 		if (len == 8 && !strncmp(spec, "builtins", 8))
-			list_builtins(0, '\n');
-		else if (len == 8 && !strncmp(spec, "parseopt", 8))
-			list_builtins(NO_PARSEOPT, ' ');
+			list_builtins(&list, 0);
 		else if (len == 4 && !strncmp(spec, "main", 4))
-			list_all_main_cmds();
+			list_all_main_cmds(&list);
 		else if (len == 6 && !strncmp(spec, "others", 6))
-			list_all_other_cmds();
+			list_all_other_cmds(&list);
+		else if (len == 6 && !strncmp(spec, "config", 6))
+			list_cmds_by_config(&list);
 		else if (len > 5 && !strncmp(spec, "list-", 5)) {
 			struct strbuf sb = STRBUF_INIT;
 
 			strbuf_add(&sb, spec + 5, len - 5);
-			list_cmds_by_category(sb.buf);
+			list_cmds_by_category(&list, sb.buf);
 			strbuf_release(&sb);
 		}
 		else
@@ -65,6 +68,9 @@ static int list_cmds(const char *spec)
 		if (*spec == ',')
 			spec++;
 	}
+	for (i = 0; i < list.nr; i++)
+		puts(list.items[i].string);
+	string_list_clear(&list, 0);
 	return 0;
 }
 
@@ -254,7 +260,18 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			(*argv)++;
 			(*argc)--;
 		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
-			exit(list_cmds(cmd));
+			if (!strcmp(cmd, "parseopt")) {
+				struct string_list list = STRING_LIST_INIT_DUP;
+				int i;
+
+				list_builtins(&list, NO_PARSEOPT);
+				for (i = 0; i < list.nr; i++)
+					printf("%s ", list.items[i].string);
+				string_list_clear(&list, 0);
+				exit(0);
+			} else {
+				exit(list_cmds(cmd));
+			}
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
 			usage(git_usage_string);
@@ -534,14 +551,14 @@ int is_builtin(const char *s)
 	return !!get_builtin(s);
 }
 
-static void list_builtins(unsigned int exclude_option, char sep)
+static void list_builtins(struct string_list *out, unsigned int exclude_option)
 {
 	int i;
 	for (i = 0; i < ARRAY_SIZE(commands); i++) {
 		if (exclude_option &&
 		    (commands[i].option & exclude_option))
 			continue;
-		printf("%s%c", commands[i].cmd, sep);
+		string_list_append(out, commands[i].cmd);
 	}
 }
 
diff --git a/help.c b/help.c
index 78ab40a0eb..abf87205b2 100644
--- a/help.c
+++ b/help.c
@@ -310,7 +310,7 @@ void list_common_cmds_help(void)
 	print_cmd_by_category(common_categories);
 }
 
-void list_all_main_cmds(void)
+void list_all_main_cmds(struct string_list *list)
 {
 	struct cmdnames main_cmds, other_cmds;
 	int i;
@@ -320,13 +320,13 @@ void list_all_main_cmds(void)
 	load_command_list("git-", &main_cmds, &other_cmds);
 
 	for (i = 0; i < main_cmds.cnt; i++)
-		puts(main_cmds.names[i]->name);
+		string_list_append(list, main_cmds.names[i]->name);
 
 	clean_cmdnames(&main_cmds);
 	clean_cmdnames(&other_cmds);
 }
 
-void list_all_other_cmds(void)
+void list_all_other_cmds(struct string_list *list)
 {
 	struct cmdnames main_cmds, other_cmds;
 	int i;
@@ -336,13 +336,14 @@ void list_all_other_cmds(void)
 	load_command_list("git-", &main_cmds, &other_cmds);
 
 	for (i = 0; i < other_cmds.cnt; i++)
-		puts(other_cmds.names[i]->name);
+		string_list_append(list, other_cmds.names[i]->name);
 
 	clean_cmdnames(&main_cmds);
 	clean_cmdnames(&other_cmds);
 }
 
-void list_cmds_by_category(const char *cat)
+void list_cmds_by_category(struct string_list *list,
+			   const char *cat)
 {
 	int i, n = ARRAY_SIZE(command_list);
 	uint32_t cat_id = 0;
@@ -359,8 +360,42 @@ void list_cmds_by_category(const char *cat)
 	for (i = 0; i < n; i++) {
 		struct cmdname_help *cmd = command_list + i;
 
-		if (cmd->category & cat_id)
-			puts(drop_prefix(cmd->name, cmd->category));
+		if (!(cmd->category & cat_id))
+			continue;
+		string_list_append(list, drop_prefix(cmd->name, cmd->category));
+	}
+}
+
+void list_cmds_by_config(struct string_list *list)
+{
+	const char *cmd_list;
+
+	/*
+	 * There's no actual repository setup at this point (and even
+	 * if there is, we don't really care; only global config
+	 * matters). If we accidentally set up a repository, it's ok
+	 * too since the caller (git --list-cmds=) should exit shortly
+	 * anyway.
+	 */
+	if (git_config_get_string_const("completion.commands", &cmd_list))
+		return;
+
+	string_list_sort(list);
+	string_list_remove_duplicates(list, 0);
+
+	while (*cmd_list) {
+		struct strbuf sb = STRBUF_INIT;
+		const char *p = strchrnul(cmd_list, ' ');
+
+		strbuf_add(&sb, cmd_list, p - cmd_list);
+		if (*cmd_list == '-')
+			string_list_remove(list, cmd_list + 1, 0);
+		else
+			string_list_insert(list, sb.buf);
+		strbuf_release(&sb);
+		while (*p == ' ')
+			p++;
+		cmd_list = p;
 	}
 }
 
diff --git a/help.h b/help.h
index 5d27368fe4..3b38292a1b 100644
--- a/help.h
+++ b/help.h
@@ -1,6 +1,8 @@
 #ifndef HELP_H
 #define HELP_H
 
+struct string_list;
+
 struct cmdnames {
 	int alloc;
 	int cnt;
@@ -19,9 +21,12 @@ static inline void mput_char(char c, unsigned int num)
 extern void list_common_cmds_help(void);
 extern void list_all_cmds_help(void);
 extern void list_common_guides_help(void);
-extern void list_all_main_cmds(void);
-extern void list_all_other_cmds(void);
-extern void list_cmds_by_category(const char *category);
+
+extern void list_all_main_cmds(struct string_list *list);
+extern void list_all_other_cmds(struct string_list *list);
+extern void list_cmds_by_category(struct string_list *list,
+				  const char *category);
+extern void list_cmds_by_config(struct string_list *list);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,

Nguyễn Thái Ngọc Duy (13):
  generate-cmds.sh: factor out synopsis extract code
  generate-cmds.sh: export all commands to command-list.h
  help: use command-list.h for common command list
  Remove common-cmds.h
  git.c: convert --list-* to --list-cmds=*
  git --list-cmds: collect command list in a string_list
  completion: implement and use --list-cmds=main,others
  git: support --list-cmds=list-<category>
  help: add "-a --verbose" to list all commands with synopsis
  help: use command-list.txt for the source of guides
  command-list.txt: documentation and guide line
  completion: let git provide the completable command list
  completion: allow to customize the completable command list

 .gitignore                             |   2 +-
 Documentation/git-help.txt             |   4 +-
 Documentation/gitattributes.txt        |   2 +-
 Documentation/gitmodules.txt           |   2 +-
 Documentation/gitrevisions.txt         |   2 +-
 Makefile                               |  16 +-
 builtin/help.c                         |  39 +---
 command-list.txt                       | 111 ++++++++---
 contrib/completion/git-completion.bash | 169 ++++++++---------
 generate-cmdlist.sh                    | 126 ++++++++-----
 git.c                                  |  61 ++++++-
 help.c                                 | 244 ++++++++++++++++++++++---
 help.h                                 |  10 +
 t/t0012-help.sh                        |  26 ++-
 t/t9902-completion.sh                  |   5 +-
 15 files changed, 565 insertions(+), 254 deletions(-)
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 01/13] generate-cmds.sh: factor out synopsis extract code

This makes it easier to reuse the same code in another place (very
soon).
---
 generate-cmdlist.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
Show changes to generate-cmdlist.sh +11 −7
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index eeea4b67ea..31b6d886cb 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,15 @@
 #!/bin/sh
 
+get_synopsis () {
+	sed -n '
+		/^NAME/,/'"$1"'/H
+		${
+			x
+			s/.*'"$1"' - \(.*\)/N_("\1")/
+			p
+		}' "Documentation/$1.txt"
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
 	char name[16];
@@ -39,12 +49,6 @@ sort |
 while read cmd tags
 do
 	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
-	sed -n '
-		/^NAME/,/git-'"$cmd"'/H
-		${
-			x
-			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
-			p
-		}' "Documentation/git-$cmd.txt"
+	echo "	{\"$cmd\", $(get_synopsis git-$cmd), $tag},"
 done
 echo "};"
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 02/13] generate-cmds.sh: export all commands to command-list.h

The current generate-cmds.sh generates just enough to print "git help" output. That is, it only extracts help text for common commands.

The script is now updated to extract help text for all commands and keep command classification a new file, command-list.h. This will be useful later:

- "git help -a" could print a short summary of all commands instead of
  just the common ones.
- "git" could produce a list of commands of one or more category. One
  of its use is to reduce another command classification embedded in
  git-completion.bash.

The new file can be generated but is not used anywhere yet. The plan is we migrate away from common-cmds.h. Then we can kill off common-cmds.h build rules and generation code (and also delete duplicate content in command-list.h which we keep for now to not mess generate-cmds.sh up too much).

PS. The new fixed column requirement on command-list.txt is
technically not needed. But it helps simplify the code a bit at this
stage. We could lift this restriction later if we want to.
---
 .gitignore          |  1 +
 Makefile            | 13 ++++++---
 command-list.txt    |  4 +--
 generate-cmdlist.sh | 67 ++++++++++++++++++++++++++++++++++++++++++---
 4 files changed, 75 insertions(+), 10 deletions(-)
Show changes to 4 files +75 −10

.gitignore, Makefile, command-list.txt, generate-cmdlist.sh

diff --git a/.gitignore b/.gitignore
index 833ef3b0b7..d4c3914167 100644
--- a/.gitignore
+++ b/.gitignore
@@ -180,6 +180,7 @@
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
 /common-cmds.h
+/command-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index f181687250..2a8913ea21 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h
+GENERATED_H += common-cmds.h command-list.h
 
 LIB_H = $(shell $(FIND) . \
 	-name .git -prune -o \
@@ -1938,6 +1938,11 @@ $(BUILT_INS): git$X
 common-cmds.h: generate-cmdlist.sh command-list.txt
 
 common-cmds.h: $(wildcard Documentation/git-*.txt)
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON >$@+ && mv $@+ $@
+
+command-list.h: generate-cmdlist.sh command-list.txt
+
+command-list.h: $(wildcard Documentation/git-*.txt)
 	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
@@ -2148,7 +2153,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h
+# Dependencies on automatically generated headers such as common-cmds.h or command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2527,7 +2532,7 @@ sparse: $(SP_OBJ)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h
+check: common-cmds.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -2775,7 +2780,7 @@ clean: profile-clean coverage-clean
 	$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/command-list.txt b/command-list.txt
index a1fad28fd8..786536aba0 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -8,8 +8,8 @@ info         examine the history and state (see also: git help revisions)
 history      grow, mark and tweak your common history
 remote       collaborate (see also: git help workflows)
 
-### command list (do not change this line)
-# command name                          category [deprecated] [common]
+### command list (do not change this line, also do not change alignment)
+# command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 31b6d886cb..c9fd524760 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,27 @@
 #!/bin/sh
 
+die () {
+	echo "$@" >&2
+	exit 1
+}
+
+command_list () {
+	sed '1,/^### command list/d;/^#/d' "$1"
+}
+
+get_categories() {
+	tr ' ' '\n'|
+	grep -v '^$' |
+	sort |
+	uniq
+}
+
+category_list () {
+	command_list "$1" |
+	cut -c 40- |
+	get_categories
+}
+
 get_synopsis () {
 	sed -n '
 		/^NAME/,/'"$1"'/H
@@ -10,14 +32,51 @@ get_synopsis () {
 		}' "Documentation/$1.txt"
 }
 
+define_categories() {
+	echo
+	echo "/* Command categories */"
+	bit=0
+	category_list "$1" |
+	while read cat
+	do
+		echo "#define CAT_$cat (1UL << $bit)"
+		bit=$(($bit+1))
+	done
+	test "$bit" -gt 32 && die "Urgh.. too many categories?"
+}
+
+print_command_list() {
+	echo "static struct cmdname_help command_list[] = {"
+
+	command_list "$1" |
+	while read cmd rest
+	do
+		printf "	{ \"$cmd\", $(get_synopsis $cmd), 0"
+		for cat in $(echo "$rest" | get_categories)
+		do
+			printf " | CAT_$cat"
+		done
+		echo " },"
+	done
+	echo "};"
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
-	char name[16];
-	char help[80];
-	unsigned char group;
+	const char *name;
+	const char *help;
+	uint32_t group;
 };
+"
+if [ -z "$2" ]
+then
+	define_categories "$1"
+	echo
+	print_command_list "$1"
+	exit 0
+fi
 
-static const char *common_cmd_groups[] = {"
+echo "static const char *common_cmd_groups[] = {"
 
 grps=grps$$.tmp
 match=match$$.tmp
-- 
2.17.0.705.g3525833791
Junio C Hamano· May 8, 2018, 03:47 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v6 02/13] generate-cmds.sh: export all commands to command-list.h

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 24 quoted lines
> The current generate-cmds.sh generates just enough to print "git help"
> output. That is, it only extracts help text for common commands.
>
> The script is now updated to extract help text for all commands and
> keep command classification a new file, command-list.h. This will be
> useful later:
>
> - "git help -a" could print a short summary of all commands instead of
>   just the common ones.
>
> - "git" could produce a list of commands of one or more category. One
>   of its use is to reduce another command classification embedded in
>   git-completion.bash.
>
> The new file can be generated but is not used anywhere yet. The plan
> is we migrate away from common-cmds.h. Then we can kill off
> common-cmds.h build rules and generation code (and also delete
> duplicate content in command-list.h which we keep for now to not mess
> generate-cmds.sh up too much).
>
> PS. The new fixed column requirement on command-list.txt is
> technically not needed. But it helps simplify the code a bit at this
> stage. We could lift this restriction later if we want to.
> ---
Missing sign-off.
Show 17 quoted lines
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index 31b6d886cb..c9fd524760 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
> @@ -1,5 +1,27 @@
>  #!/bin/sh
>  
> +die () {
> +	echo "$@" >&2
> +	exit 1
> +}
> +
> +command_list () {
> +	sed '1,/^### command list/d;/^#/d' "$1"
> +}
> +
> +get_categories() {

Here and define_categories() below lack SP before (), and all others do. Be consistent.

Show 26 quoted lines
> +	tr ' ' '\n'|
> +	grep -v '^$' |
> +	sort |
> +	uniq
> +}
> +
> +category_list () {
> +	command_list "$1" |
> +	cut -c 40- |
> +	get_categories
> +}
> +
>  get_synopsis () {
>  	sed -n '
>  		/^NAME/,/'"$1"'/H
> @@ -10,14 +32,51 @@ get_synopsis () {
>  		}' "Documentation/$1.txt"
>  }
>  
> +define_categories() {
> +	echo
> +	echo "/* Command categories */"
> +...
>  };
> +"
> +if [ -z "$2" ]
if test -z "$2"
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 04/13] Remove common-cmds.h

After the last patch, common-cmds.h is no longer used (and it was
actually broken). Remove all related code. command-list.h will take
its place from now on.
---
 .gitignore          |  1 -
 Makefile            | 17 ++++++-----------
 generate-cmdlist.sh | 46 +++------------------------------------------
 3 files changed, 9 insertions(+), 55 deletions(-)
Show changes to 3 files +9 −55

.gitignore, Makefile, generate-cmdlist.sh

diff --git a/.gitignore b/.gitignore
index d4c3914167..0836083992 100644
--- a/.gitignore
+++ b/.gitignore
@@ -179,7 +179,6 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
-/common-cmds.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 5c58b0b692..a60a78ee67 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h command-list.h
+GENERATED_H += command-list.h
 
 LIB_H = $(shell $(FIND) . \
 	-name .git -prune -o \
@@ -1914,9 +1914,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
-help.sp help.s help.o: common-cmds.h command-list.h
+help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -1935,11 +1935,6 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
-common-cmds.h: generate-cmdlist.sh command-list.txt
-
-common-cmds.h: $(wildcard Documentation/git-*.txt)
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON >$@+ && mv $@+ $@
-
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git-*.txt)
@@ -2153,7 +2148,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h or command-list.h
+# Dependencies on automatically generated headers such as command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2532,7 +2527,7 @@ sparse: $(SP_OBJ)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h command-list.h
+check: command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -2780,7 +2775,7 @@ clean: profile-clean coverage-clean
 	$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 93de8e8f59..015eef2804 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -68,46 +68,6 @@ struct cmdname_help {
 	uint32_t category;
 };
 "
-if [ -z "$2" ]
-then
-	define_categories "$1"
-	echo
-	print_command_list "$1"
-	exit 0
-fi
-
-echo "static const char *common_cmd_groups[] = {"
-
-grps=grps$$.tmp
-match=match$$.tmp
-trap "rm -f '$grps' '$match'" 0 1 2 3 15
-
-sed -n '
-	1,/^### common groups/b
-	/^### command list/q
-	/^#/b
-	/^[ 	]*$/b
-	h;s/^[^ 	][^ 	]*[ 	][ 	]*\(.*\)/	N_("\1"),/p
-	g;s/^\([^ 	][^ 	]*\)[ 	].*/\1/w '$grps'
-	' "$1"
-printf '};\n\n'
-
-n=0
-substnum=
-while read grp
-do
-	echo "^git-..*[ 	]$grp"
-	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
-	n=$(($n+1))
-done <"$grps" >"$match"
-
-printf 'static struct cmdname_help common_cmds[] = {\n'
-grep -f "$match" "$1" |
-sed 's/^git-//' |
-sort |
-while read cmd tags
-do
-	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
-	echo "	{\"$cmd\", $(get_synopsis git-$cmd), $tag},"
-done
-echo "};"
+define_categories "$1"
+echo
+print_command_list "$1"
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 06/13] git --list-cmds: collect command list in a string_list

Instead of printing the command directly one by one, keep them in a
list and print at the end. This allows more modification before we
print out (e.g. sorting, removing duplicates or even excluding some
items).
---
 git.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)
Show changes to git.c +17 −5
diff --git a/git.c b/git.c
index b2842a22e2..a0477c459e 100644
--- a/git.c
+++ b/git.c
@@ -36,22 +36,28 @@ const char git_more_info_string[] =
 
 static int use_pager = -1;
 
-static void list_builtins(unsigned int exclude_option, char sep);
+static void list_builtins(struct string_list *list, unsigned int exclude_option);
 
 static int list_cmds(const char *spec)
 {
+	struct string_list list = STRING_LIST_INIT_DUP;
+	int i;
+
 	while (*spec) {
 		const char *sep = strchrnul(spec, ',');
 		int len = sep - spec;
 
 		if (len == 8 && !strncmp(spec, "builtins", 8))
-			list_builtins(0, '\n');
+			list_builtins(&list, 0);
 		else
 			die(_("unsupported command listing type '%s'"), spec);
 		spec += len;
 		if (*spec == ',')
 			spec++;
 	}
+	for (i = 0; i < list.nr; i++)
+		puts(list.items[i].string);
+	string_list_clear(&list, 0);
 	return 0;
 }
 
@@ -242,7 +248,13 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			(*argc)--;
 		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
 			if (!strcmp(cmd, "parseopt")) {
-				list_builtins(NO_PARSEOPT, ' ');
+				struct string_list list = STRING_LIST_INIT_DUP;
+				int i;
+
+				list_builtins(&list, NO_PARSEOPT);
+				for (i = 0; i < list.nr; i++)
+					printf("%s ", list.items[i].string);
+				string_list_clear(&list, 0);
 				exit(0);
 			} else {
 				exit(list_cmds(cmd));
@@ -526,14 +538,14 @@ int is_builtin(const char *s)
 	return !!get_builtin(s);
 }
 
-static void list_builtins(unsigned int exclude_option, char sep)
+static void list_builtins(struct string_list *out, unsigned int exclude_option)
 {
 	int i;
 	for (i = 0; i < ARRAY_SIZE(commands); i++) {
 		if (exclude_option &&
 		    (commands[i].option & exclude_option))
 			continue;
-		printf("%s%c", commands[i].cmd, sep);
+		string_list_append(out, commands[i].cmd);
 	}
 }
 
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 07/13] completion: implement and use --list-cmds=main,others

Instead of parsing "git help -a" output, which is tricky to get right,
less elegant and also slow, make git provide the list in a
machine-friendly form. This adds two separate listing types, main and
others, instead of just "all" for more flexibility.
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  |  4 ++++
 help.c                                 | 32 ++++++++++++++++++++++++++
 help.h                                 |  4 ++++
 4 files changed, 41 insertions(+), 1 deletion(-)
Show changes to 4 files +41 −1

contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3556838759..62ca8641f4 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -839,7 +839,7 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git help -a|egrep '^  [a-zA-Z0-9]'
+		git --list-cmds=main,others
 	fi
 }
 
diff --git a/git.c b/git.c
index a0477c459e..3c032d01fc 100644
--- a/git.c
+++ b/git.c
@@ -49,6 +49,10 @@ static int list_cmds(const char *spec)
 
 		if (len == 8 && !strncmp(spec, "builtins", 8))
 			list_builtins(&list, 0);
+		else if (len == 4 && !strncmp(spec, "main", 4))
+			list_all_main_cmds(&list);
+		else if (len == 6 && !strncmp(spec, "others", 6))
+			list_all_other_cmds(&list);
 		else
 			die(_("unsupported command listing type '%s'"), spec);
 		spec += len;
diff --git a/help.c b/help.c
index 2d6a3157f8..d5ce9dfcbb 100644
--- a/help.c
+++ b/help.c
@@ -297,6 +297,38 @@ void list_common_cmds_help(void)
 	print_cmd_by_category(common_categories);
 }
 
+void list_all_main_cmds(struct string_list *list)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < main_cmds.cnt; i++)
+		string_list_append(list, main_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
+void list_all_other_cmds(struct string_list *list)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < other_cmds.cnt; i++)
+		string_list_append(list, other_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index b21d7c94e8..97e6c0965e 100644
--- a/help.h
+++ b/help.h
@@ -1,6 +1,8 @@
 #ifndef HELP_H
 #define HELP_H
 
+struct string_list;
+
 struct cmdnames {
 	int alloc;
 	int cnt;
@@ -17,6 +19,8 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_main_cmds(struct string_list *list);
+extern void list_all_other_cmds(struct string_list *list);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 05/13] git.c: convert --list-* to --list-cmds=*

Even if these are hidden options, let's make them a bit more generic since we're introducing more listing types shortly. The code is structured to allow combining multiple listing types together because we will soon add more types the 'builtins'.

'parseopt' remains separate because it has separate (SPC) to match
git-completion.bash needs and will not combine with others.
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  | 30 ++++++++++++++++++++------
 t/t0012-help.sh                        |  2 +-
 3 files changed, 26 insertions(+), 8 deletions(-)
Show changes to 3 files +26 −8

contrib/completion/git-completion.bash, git.c, t/t0012-help.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a757073945..3556838759 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -3049,7 +3049,7 @@ __git_complete_common () {
 __git_cmds_with_parseopt_helper=
 __git_support_parseopt_helper () {
 	test -n "$__git_cmds_with_parseopt_helper" ||
-		__git_cmds_with_parseopt_helper="$(__git --list-parseopt-builtins)"
+		__git_cmds_with_parseopt_helper="$(__git --list-cmds=parseopt)"
 
 	case " $__git_cmds_with_parseopt_helper " in
 	*" $1 "*)
diff --git a/git.c b/git.c
index 3a89893712..b2842a22e2 100644
--- a/git.c
+++ b/git.c
@@ -38,6 +38,23 @@ static int use_pager = -1;
 
 static void list_builtins(unsigned int exclude_option, char sep);
 
+static int list_cmds(const char *spec)
+{
+	while (*spec) {
+		const char *sep = strchrnul(spec, ',');
+		int len = sep - spec;
+
+		if (len == 8 && !strncmp(spec, "builtins", 8))
+			list_builtins(0, '\n');
+		else
+			die(_("unsupported command listing type '%s'"), spec);
+		spec += len;
+		if (*spec == ',')
+			spec++;
+	}
+	return 0;
+}
+
 static void commit_pager_choice(void) {
 	switch (use_pager) {
 	case 0:
@@ -223,12 +240,13 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			}
 			(*argv)++;
 			(*argc)--;
-		} else if (!strcmp(cmd, "--list-builtins")) {
-			list_builtins(0, '\n');
-			exit(0);
-		} else if (!strcmp(cmd, "--list-parseopt-builtins")) {
-			list_builtins(NO_PARSEOPT, ' ');
-			exit(0);
+		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
+			if (!strcmp(cmd, "parseopt")) {
+				list_builtins(NO_PARSEOPT, ' ');
+				exit(0);
+			} else {
+				exit(list_cmds(cmd));
+			}
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
 			usage(git_usage_string);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index c096f33505..3c91a9024a 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -59,7 +59,7 @@ test_expect_success 'git help' '
 '
 
 test_expect_success 'generate builtin list' '
-	git --list-builtins >builtins
+	git --list-cmds=builtins >builtins
 '
 
 while read builtin
-- 
2.17.0.705.g3525833791
Junio C Hamano· May 8, 2018, 03:50 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v6 05/13] git.c: convert --list-* to --list-cmds=*

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 8 quoted lines
> Even if these are hidden options, let's make them a bit more generic
> since we're introducing more listing types shortly. The code is
> structured to allow combining multiple listing types together because
> we will soon add more types the 'builtins'.
>
> 'parseopt' remains separate because it has separate (SPC) to match
> git-completion.bash needs and will not combine with others.
> ---
Missing sign-off.
Show 8 quoted lines
> +static int list_cmds(const char *spec)
> +{
> +	while (*spec) {
> +		const char *sep = strchrnul(spec, ',');
> +		int len = sep - spec;
> +
> +		if (len == 8 && !strncmp(spec, "builtins", 8))
> +			list_builtins(0, '\n');

This is the origin of ugliness we see in later steps that follow the same

	if (len == strlen(constS) && !strncmp(spec, constS, strlen(constS))

pattern added here. Could we have a small helper that takes len, spec, and constS to abstract "8" away?

Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 09/13] help: add "-a --verbose" to list all commands with synopsis

This lists all recognized commands [1] by category. The group order follows closely git.txt.

[1] We may actually show commands that are not built (e.g. if you set
NO_PERL you don't have git-instaweb but it's still listed here). I
ignore the problem because on Linux a git package could be split
anyway. The "git-core" package may not contain git-instaweb even if
it's built because it may end up in a separate package. We can't know
anyway.
---
 Documentation/git-help.txt |  4 +++-
 builtin/help.c             |  7 +++++++
 help.c                     | 16 ++++++++++++++++
 help.h                     |  2 ++
 t/t0012-help.sh            |  9 +++++++++
 5 files changed, 37 insertions(+), 1 deletion(-)
Show changes to 5 files +37 −1

Documentation/git-help.txt, builtin/help.c, help.c, help.h, t/t0012-help.sh

diff --git a/Documentation/git-help.txt b/Documentation/git-help.txt
index 40d328a4b3..a40fc38d8b 100644
--- a/Documentation/git-help.txt
+++ b/Documentation/git-help.txt
@@ -8,7 +8,7 @@ git-help - Display help information about Git
 SYNOPSIS
 --------
 [verse]
-'git help' [-a|--all] [-g|--guide]
+'git help' [-a|--all [--verbose]] [-g|--guide]
 	   [-i|--info|-m|--man|-w|--web] [COMMAND|GUIDE]
 
 DESCRIPTION
@@ -42,6 +42,8 @@ OPTIONS
 --all::
 	Prints all the available commands on the standard output. This
 	option overrides any given command or guide name.
+	When used with `--verbose` print description for all recognized
+	commands.
 
 -g::
 --guides::
diff --git a/builtin/help.c b/builtin/help.c
index 598867cfea..0e0af8426a 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -36,6 +36,7 @@ static const char *html_path;
 
 static int show_all = 0;
 static int show_guides = 0;
+static int verbose;
 static unsigned int colopts;
 static enum help_format help_format = HELP_FORMAT_NONE;
 static int exclude_guides;
@@ -48,6 +49,7 @@ static struct option builtin_help_options[] = {
 			HELP_FORMAT_WEB),
 	OPT_SET_INT('i', "info", &help_format, N_("show info page"),
 			HELP_FORMAT_INFO),
+	OPT__VERBOSE(&verbose, N_("print command description")),
 	OPT_END(),
 };
 
@@ -463,6 +465,11 @@ int cmd_help(int argc, const char **argv, const char *prefix)
 
 	if (show_all) {
 		git_config(git_help_config, NULL);
+		if (verbose) {
+			setup_pager();
+			list_all_cmds_help();
+			return 0;
+		}
 		printf(_("usage: %s%s"), _(git_usage_string), "\n\n");
 		load_command_list("git-", &main_cmds, &other_cmds);
 		list_commands(colopts, &main_cmds, &other_cmds);
diff --git a/help.c b/help.c
index 1117f7d1d1..c7df1d2338 100644
--- a/help.c
+++ b/help.c
@@ -27,6 +27,17 @@ static struct category_description common_categories[] = {
 	{ CAT_remote, N_("collaborate (see also: git help workflows)") },
 	{ 0, NULL }
 };
+static struct category_description main_categories[] = {
+	{ CAT_mainporcelain, N_("Main Porcelain Commands") },
+	{ CAT_ancillarymanipulators, N_("Ancillary Commands / Manipulators") },
+	{ CAT_ancillaryinterrogators, N_("Ancillary Commands / Interrogators") },
+	{ CAT_foreignscminterface, N_("Interacting with Others") },
+	{ CAT_plumbingmanipulators, N_("Low-level Commands / Manipulators") },
+	{ CAT_plumbinginterrogators, N_("Low-level Commands / Interrogators") },
+	{ CAT_synchingrepositories, N_("Low-level Commands / Synching Repositories") },
+	{ CAT_purehelpers, N_("Low-level Commands / Internal Helpers") },
+	{ 0, NULL }
+};
 
 static const char *drop_prefix(const char *name)
 {
@@ -352,6 +363,11 @@ void list_cmds_by_category(struct string_list *list,
 	}
 }
 
+void list_all_cmds_help(void)
+{
+	print_cmd_by_category(main_categories);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 734bba32d3..40917fc38c 100644
--- a/help.h
+++ b/help.h
@@ -19,6 +19,8 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds_help(void);
+
 extern void list_all_main_cmds(struct string_list *list);
 extern void list_all_other_cmds(struct string_list *list);
 extern void list_cmds_by_category(struct string_list *list,
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 3c91a9024a..060df24c2d 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -25,6 +25,15 @@ test_expect_success "setup" '
 	EOF
 '
 
+# make sure to exercise these code paths, the output is a bit tricky
+# to verify
+test_expect_success 'basic help commands' '
+	git help >/dev/null &&
+	git help -a >/dev/null &&
+	git help -g >/dev/null &&
+	git help -av >/dev/null
+'
+
 test_expect_success "works for commands and guides by default" '
 	configure_help &&
 	git help status &&
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 12/13] completion: let git provide the completable command list

Instead of maintaining a separate list of command classification, which often could go out of date, let's centralize the information back in git.

While the function in git-completion.bash implies "list porcelain commands", that's not exactly what it does. It gets all commands (aka --list-cmds=main,others) then exclude certain non-porcelain ones. We could almost recreate this list two lists list-mainporcelain and others. The non-porcelain-but-included-anyway is added by the third category list-complete.

list-complete does not recreate exactly the command list before this patch though. The following commands are not part of neither list-mainporcelain nor list-complete and as a result no longer completes:

- annotate             obsolete, discouraged to use
- difftool-helper      not an end user command
- filter-branch        not often used
- get-tar-commit-id    not often used
- imap-send            not often used
- interpreter-trailers not for interactive use
- lost-found           obsolete
- p4                   too short and probably not often used (*)
- peek-remote          deprecated
- svn                  same category as p4 (*)
- tar-tree             obsolete
- verify-commit        not often used

Note that the current completion script incorrectly classifies filter-branch as porcelain and t9902 tests this behavior. We keep it this way in t9902 because this test does not really care which particular command is porcelain or plubmbing, they're just names.

(*) to be fair, send-email command which is in the same
foreignscminterface group as svn and p4 does get completion, just
because it's used by git and kernel development. So maybe should
include them.
---
 command-list.txt                       |  37 ++++----
 contrib/completion/git-completion.bash | 117 ++++++-------------------
 t/t9902-completion.sh                  |   5 +-
 3 files changed, 48 insertions(+), 111 deletions(-)
Show changes to 3 files +48 −111

command-list.txt, contrib/completion/git-completion.bash, t/t9902-completion.sh

diff --git a/command-list.txt b/command-list.txt
index 9c70c69193..3e21ddfcfb 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -47,11 +47,11 @@
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
-git-apply                               plumbingmanipulators
+git-apply                               plumbingmanipulators            complete
 git-archimport                          foreignscminterface
 git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
-git-blame                               ancillaryinterrogators
+git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
@@ -61,7 +61,7 @@ git-check-mailmap                       purehelpers
 git-checkout                            mainporcelain           history
 git-checkout-index                      plumbingmanipulators
 git-check-ref-format                    purehelpers
-git-cherry                              ancillaryinterrogators
+git-cherry                              ancillaryinterrogators          complete
 git-cherry-pick                         mainporcelain
 git-citool                              mainporcelain
 git-clean                               mainporcelain
@@ -69,7 +69,7 @@ git-clone                               mainporcelain           init
 git-column                              purehelpers
 git-commit                              mainporcelain           history
 git-commit-tree                         plumbingmanipulators
-git-config                              ancillarymanipulators
+git-config                              ancillarymanipulators           complete
 git-count-objects                       ancillaryinterrogators
 git-credential                          purehelpers
 git-credential-cache                    purehelpers
@@ -83,7 +83,7 @@ git-diff                                mainporcelain           history
 git-diff-files                          plumbinginterrogators
 git-diff-index                          plumbinginterrogators
 git-diff-tree                           plumbinginterrogators
-git-difftool                            ancillaryinterrogators
+git-difftool                            ancillaryinterrogators          complete
 git-fast-export                         ancillarymanipulators
 git-fast-import                         ancillarymanipulators
 git-fetch                               mainporcelain           remote
@@ -92,20 +92,20 @@ git-filter-branch                       ancillarymanipulators
 git-fmt-merge-msg                       purehelpers
 git-for-each-ref                        plumbinginterrogators
 git-format-patch                        mainporcelain
-git-fsck                                ancillaryinterrogators
+git-fsck                                ancillaryinterrogators          complete
 git-gc                                  mainporcelain
 git-get-tar-commit-id                   ancillaryinterrogators
 git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
-git-help                                ancillaryinterrogators
+git-help                                ancillaryinterrogators          complete
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
 git-imap-send                           foreignscminterface
 git-index-pack                          plumbingmanipulators
 git-init                                mainporcelain           init
-git-instaweb                            ancillaryinterrogators
+git-instaweb                            ancillaryinterrogators          complete
 git-interpret-trailers                  purehelpers
 gitk                                    mainporcelain
 git-log                                 mainporcelain           info
@@ -119,12 +119,12 @@ git-merge-base                          plumbinginterrogators
 git-merge-file                          plumbingmanipulators
 git-merge-index                         plumbingmanipulators
 git-merge-one-file                      purehelpers
-git-mergetool                           ancillarymanipulators
+git-mergetool                           ancillarymanipulators           complete
 git-merge-tree                          ancillaryinterrogators
 git-mktag                               plumbingmanipulators
 git-mktree                              plumbingmanipulators
 git-mv                                  mainporcelain           worktree
-git-name-rev                            plumbinginterrogators
+git-name-rev                            plumbinginterrogators           complete
 git-notes                               mainporcelain
 git-p4                                  foreignscminterface
 git-pack-objects                        plumbingmanipulators
@@ -140,28 +140,29 @@ git-quiltimport                         foreignscminterface
 git-read-tree                           plumbingmanipulators
 git-rebase                              mainporcelain           history
 git-receive-pack                        synchelpers
-git-reflog                              ancillarymanipulators
-git-remote                              ancillarymanipulators
-git-repack                              ancillarymanipulators
-git-replace                             ancillarymanipulators
-git-request-pull                        foreignscminterface
+git-reflog                              ancillarymanipulators           complete
+git-remote                              ancillarymanipulators           complete
+git-repack                              ancillarymanipulators           complete
+git-replace                             ancillarymanipulators           complete
+git-request-pull                        foreignscminterface             complete
 git-rerere                              ancillaryinterrogators
 git-reset                               mainporcelain           worktree
 git-revert                              mainporcelain
 git-rev-list                            plumbinginterrogators
 git-rev-parse                           ancillaryinterrogators
 git-rm                                  mainporcelain           worktree
-git-send-email                          foreignscminterface
+git-send-email                          foreignscminterface             complete
 git-send-pack                           synchingrepositories
 git-shell                               synchelpers
 git-shortlog                            mainporcelain
 git-show                                mainporcelain           info
-git-show-branch                         ancillaryinterrogators
+git-show-branch                         ancillaryinterrogators          complete
 git-show-index                          plumbinginterrogators
 git-show-ref                            plumbinginterrogators
 git-sh-i18n                             purehelpers
 git-sh-setup                            purehelpers
 git-stash                               mainporcelain
+git-stage                                                               complete
 git-status                              mainporcelain           info
 git-stripspace                          purehelpers
 git-submodule                           mainporcelain
@@ -180,7 +181,7 @@ git-verify-commit                       ancillaryinterrogators
 git-verify-pack                         plumbinginterrogators
 git-verify-tag                          ancillaryinterrogators
 gitweb                                  ancillaryinterrogators
-git-whatchanged                         ancillaryinterrogators
+git-whatchanged                         ancillaryinterrogators          complete
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
 gitattributes                           guide
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 4e724a5b76..908692ea52 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -835,18 +835,30 @@ __git_complete_strategy ()
 }
 
 __git_commands () {
-	if test -n "${GIT_TESTING_COMMAND_COMPLETION:-}"
-	then
-		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
-	else
-		git --list-cmds=main,others
-	fi
+	case "$1" in
+	porcelain)
+		if test -n "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		then
+			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		else
+			git --list-cmds=list-mainporcelain,others,list-complete
+		fi
+		;;
+	all)
+		if test -n "$GIT_TESTING_ALL_COMMAND_LIST"
+		then
+			printf "%s" "$GIT_TESTING_ALL_COMMAND_LIST"
+		else
+			git --list-cmds=main,others
+		fi
+		;;
+	esac
 }
 
-__git_list_all_commands ()
+__git_list_commands ()
 {
 	local i IFS=" "$'\n'
-	for i in $(__git_commands)
+	for i in $(__git_commands $1)
 	do
 		case $i in
 		*--*)             : helper pattern;;
@@ -855,6 +867,11 @@ __git_list_all_commands ()
 	done
 }
 
+__git_list_all_commands ()
+{
+	__git_list_commands all
+}
+
 __git_all_commands=
 __git_compute_all_commands ()
 {
@@ -864,89 +881,7 @@ __git_compute_all_commands ()
 
 __git_list_porcelain_commands ()
 {
-	local i IFS=" "$'\n'
-	__git_compute_all_commands
-	for i in $__git_all_commands
-	do
-		case $i in
-		*--*)             : helper pattern;;
-		applymbox)        : ask gittus;;
-		applypatch)       : ask gittus;;
-		archimport)       : import;;
-		cat-file)         : plumbing;;
-		check-attr)       : plumbing;;
-		check-ignore)     : plumbing;;
-		check-mailmap)    : plumbing;;
-		check-ref-format) : plumbing;;
-		checkout-index)   : plumbing;;
-		column)           : internal helper;;
-		commit-tree)      : plumbing;;
-		count-objects)    : infrequent;;
-		credential)       : credentials;;
-		credential-*)     : credentials helper;;
-		cvsexportcommit)  : export;;
-		cvsimport)        : import;;
-		cvsserver)        : daemon;;
-		daemon)           : daemon;;
-		diff-files)       : plumbing;;
-		diff-index)       : plumbing;;
-		diff-tree)        : plumbing;;
-		fast-import)      : import;;
-		fast-export)      : export;;
-		fsck-objects)     : plumbing;;
-		fetch-pack)       : plumbing;;
-		fmt-merge-msg)    : plumbing;;
-		for-each-ref)     : plumbing;;
-		hash-object)      : plumbing;;
-		http-*)           : transport;;
-		index-pack)       : plumbing;;
-		init-db)          : deprecated;;
-		local-fetch)      : plumbing;;
-		ls-files)         : plumbing;;
-		ls-remote)        : plumbing;;
-		ls-tree)          : plumbing;;
-		mailinfo)         : plumbing;;
-		mailsplit)        : plumbing;;
-		merge-*)          : plumbing;;
-		mktree)           : plumbing;;
-		mktag)            : plumbing;;
-		pack-objects)     : plumbing;;
-		pack-redundant)   : plumbing;;
-		pack-refs)        : plumbing;;
-		parse-remote)     : plumbing;;
-		patch-id)         : plumbing;;
-		prune)            : plumbing;;
-		prune-packed)     : plumbing;;
-		quiltimport)      : import;;
-		read-tree)        : plumbing;;
-		receive-pack)     : plumbing;;
-		remote-*)         : transport;;
-		rerere)           : plumbing;;
-		rev-list)         : plumbing;;
-		rev-parse)        : plumbing;;
-		runstatus)        : plumbing;;
-		sh-setup)         : internal;;
-		shell)            : daemon;;
-		show-ref)         : plumbing;;
-		send-pack)        : plumbing;;
-		show-index)       : plumbing;;
-		ssh-*)            : transport;;
-		stripspace)       : plumbing;;
-		symbolic-ref)     : plumbing;;
-		unpack-file)      : plumbing;;
-		unpack-objects)   : plumbing;;
-		update-index)     : plumbing;;
-		update-ref)       : plumbing;;
-		update-server-info) : daemon;;
-		upload-archive)   : plumbing;;
-		upload-pack)      : plumbing;;
-		write-tree)       : plumbing;;
-		var)              : infrequent;;
-		verify-pack)      : infrequent;;
-		verify-tag)       : plumbing;;
-		*) echo $i;;
-		esac
-	done
+	__git_list_commands porcelain
 }
 
 __git_porcelain_commands=
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 1b34caa1e1..2f16679380 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -13,7 +13,7 @@ complete ()
 	return 0
 }
 
-# Be careful when updating this list:
+# Be careful when updating these lists:
 #
 # (1) The build tree may have build artifact from different branch, or
 #     the user's $PATH may have a random executable that may begin
@@ -30,7 +30,8 @@ complete ()
 #     completion for "git <TAB>", and a plumbing is excluded.  "add",
 #     "filter-branch" and "ls-files" are listed for this.
 
-GIT_TESTING_COMMAND_COMPLETION='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_ALL_COMMAND_LIST='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_PORCELAIN_COMMAND_LIST='add checkout filter-branch'
 
 . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash"
 
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 13/13] completion: allow to customize the completable command list

By default we show porcelain, external commands and a couple others that are also popular. If you are not happy with this list, you can now customize it. See the big comment block for details.

PS. perhaps I should make aliases a group too, which makes it possible
to _not_ complete aliases by omitting this special group in
$GIT_COMPLETION_CMD_GROUPS
---
 contrib/completion/git-completion.bash | 35 ++++++++++++++++++++++++++
 git.c                                  |  2 ++
 help.c                                 | 33 ++++++++++++++++++++++++
 help.h                                 |  1 +
 4 files changed, 71 insertions(+)
Show changes to 4 files +71 −0

contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 908692ea52..0fd29803d5 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -38,6 +38,38 @@
 #
 #     When set to "1", do not include "DWIM" suggestions in git-checkout
 #     completion (e.g., completing "foo" when "origin/foo" exists).
+#
+#   GIT_COMPLETION_CMD_GROUPS
+#
+#     When set, "git --list-cmds=$GIT_COMPLETION_CMD_GROUPS" will be
+#     used to get the list of completable commands. The default is
+#     "mainporcelain,others,list-complete" (in English: all porcelain
+#     commands and external ones are included. Certain non-porcelain
+#     commands are also marked for completion in command-list.txt).
+#     You could for example complete all commands with
+#
+#         GIT_COMPLETION_CMD_GROUPS=main,others
+#
+#     Or you could go with defaults add some extra commands specified
+#     in the configuration variable completion.commands [1] with
+#
+#         GIT_COMPLETION_CMD_GROUPS=mainporcelain,others,list-complete,config
+#
+#     Or go completely custom group with
+#
+#         GIT_COMPLETION_CMD_GROUPS=config
+#
+#     Or you could even play with other command categories found in
+#     command-list.txt.
+#
+#     [1] Note that completion.commands should not be per-repository
+#         since the command list is generated once and cached.
+#
+#         completion.commands could be used to exclude commands as
+#         well.  If a command in this list begins with '-', then it
+#         will be excluded from the list of commands gathered by the
+#         groups specified before "config" in
+#         $GIT_COMPLETION_CMD_GROUPS.
 
 case "$COMP_WORDBREAKS" in
 *:*) : great ;;
@@ -840,6 +872,9 @@ __git_commands () {
 		if test -n "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
 		then
 			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		elif test -n "$GIT_COMPLETION_CMD_GROUPS"
+		then
+			git --list-cmds="$GIT_COMPLETION_CMD_GROUPS"
 		else
 			git --list-cmds=list-mainporcelain,others,list-complete
 		fi
diff --git a/git.c b/git.c
index 67f3e20ae9..fd08911e11 100644
--- a/git.c
+++ b/git.c
@@ -53,6 +53,8 @@ static int list_cmds(const char *spec)
 			list_all_main_cmds(&list);
 		else if (len == 6 && !strncmp(spec, "others", 6))
 			list_all_other_cmds(&list);
+		else if (len == 6 && !strncmp(spec, "config", 6))
+			list_cmds_by_config(&list);
 		else if (len > 5 && !strncmp(spec, "list-", 5)) {
 			struct strbuf sb = STRBUF_INIT;
 
diff --git a/help.c b/help.c
index 23924dd300..abf87205b2 100644
--- a/help.c
+++ b/help.c
@@ -366,6 +366,39 @@ void list_cmds_by_category(struct string_list *list,
 	}
 }
 
+void list_cmds_by_config(struct string_list *list)
+{
+	const char *cmd_list;
+
+	/*
+	 * There's no actual repository setup at this point (and even
+	 * if there is, we don't really care; only global config
+	 * matters). If we accidentally set up a repository, it's ok
+	 * too since the caller (git --list-cmds=) should exit shortly
+	 * anyway.
+	 */
+	if (git_config_get_string_const("completion.commands", &cmd_list))
+		return;
+
+	string_list_sort(list);
+	string_list_remove_duplicates(list, 0);
+
+	while (*cmd_list) {
+		struct strbuf sb = STRBUF_INIT;
+		const char *p = strchrnul(cmd_list, ' ');
+
+		strbuf_add(&sb, cmd_list, p - cmd_list);
+		if (*cmd_list == '-')
+			string_list_remove(list, cmd_list + 1, 0);
+		else
+			string_list_insert(list, sb.buf);
+		strbuf_release(&sb);
+		while (*p == ' ')
+			p++;
+		cmd_list = p;
+	}
+}
+
 void list_common_guides_help(void)
 {
 	struct category_description catdesc[] = {
diff --git a/help.h b/help.h
index b2293e99be..3b38292a1b 100644
--- a/help.h
+++ b/help.h
@@ -26,6 +26,7 @@ extern void list_all_main_cmds(struct string_list *list);
 extern void list_all_other_cmds(struct string_list *list);
 extern void list_cmds_by_category(struct string_list *list,
 				  const char *category);
+extern void list_cmds_by_config(struct string_list *list);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 10/13] help: use command-list.txt for the source of guides

The help command currently hard codes the list of guides and their
summary in C. Let's move this list to command-list.txt. This lets us
extract summary lines from Documentation/git*.txt. This also
potentially lets us list guides in git.txt, but I'll leave that for
now.
---
 Documentation/gitattributes.txt        |  2 +-
 Documentation/gitmodules.txt           |  2 +-
 Documentation/gitrevisions.txt         |  2 +-
 Makefile                               |  2 +-
 builtin/help.c                         | 32 --------------------------
 command-list.txt                       | 16 +++++++++++++
 contrib/completion/git-completion.bash | 15 ++++++++----
 help.c                                 | 21 +++++++++++++----
 help.h                                 |  1 +
 t/t0012-help.sh                        |  6 +++++
 10 files changed, 54 insertions(+), 45 deletions(-)
Show changes to 10 files +54 −45

Documentation/gitattributes.txt, Documentation/gitmodules.txt, Documentation/gitrevisions.txt, Makefile, builtin/help.c, command-list.txt, contrib/completion/git-completion.bash, help.c, help.h, t/t0012-help.sh

diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 1094fe2b5b..083c2f380d 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -3,7 +3,7 @@ gitattributes(5)
 
 NAME
 ----
-gitattributes - defining attributes per path
+gitattributes - Defining attributes per path
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index db5d47eb19..4d63def206 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -3,7 +3,7 @@ gitmodules(5)
 
 NAME
 ----
-gitmodules - defining submodule properties
+gitmodules - Defining submodule properties
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitrevisions.txt b/Documentation/gitrevisions.txt
index 27dec5b91d..1f6cceaefb 100644
--- a/Documentation/gitrevisions.txt
+++ b/Documentation/gitrevisions.txt
@@ -3,7 +3,7 @@ gitrevisions(7)
 
 NAME
 ----
-gitrevisions - specifying revisions and ranges for Git
+gitrevisions - Specifying revisions and ranges for Git
 
 SYNOPSIS
 --------
diff --git a/Makefile b/Makefile
index a60a78ee67..1efb751e46 100644
--- a/Makefile
+++ b/Makefile
@@ -1937,7 +1937,7 @@ $(BUILT_INS): git$X
 
 command-list.h: generate-cmdlist.sh command-list.txt
 
-command-list.h: $(wildcard Documentation/git-*.txt)
+command-list.h: $(wildcard Documentation/git*.txt)
 	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
diff --git a/builtin/help.c b/builtin/help.c
index 0e0af8426a..5727fb5e51 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -402,38 +402,6 @@ static void show_html_page(const char *git_cmd)
 	open_html(page_path.buf);
 }
 
-static struct {
-	const char *name;
-	const char *help;
-} common_guides[] = {
-	{ "attributes", N_("Defining attributes per path") },
-	{ "everyday", N_("Everyday Git With 20 Commands Or So") },
-	{ "glossary", N_("A Git glossary") },
-	{ "ignore", N_("Specifies intentionally untracked files to ignore") },
-	{ "modules", N_("Defining submodule properties") },
-	{ "revisions", N_("Specifying revisions and ranges for Git") },
-	{ "tutorial", N_("A tutorial introduction to Git (for version 1.5.1 or newer)") },
-	{ "workflows", N_("An overview of recommended workflows with Git") },
-};
-
-static void list_common_guides_help(void)
-{
-	int i, longest = 0;
-
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		if (longest < strlen(common_guides[i].name))
-			longest = strlen(common_guides[i].name);
-	}
-
-	puts(_("The common Git guides are:\n"));
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		printf("   %s   ", common_guides[i].name);
-		mput_char(' ', longest - strlen(common_guides[i].name));
-		puts(_(common_guides[i].help));
-	}
-	putchar('\n');
-}
-
 static const char *check_git_cmd(const char* cmd)
 {
 	char *alias;
diff --git a/command-list.txt b/command-list.txt
index 3bd23201a6..99ddc231c1 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -139,3 +139,19 @@ gitweb                                  ancillaryinterrogators
 git-whatchanged                         ancillaryinterrogators
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
+gitattributes                           guide
+gitcli                                  guide
+gitcore-tutorial                        guide
+gitcvs-migration                        guide
+gitdiffcore                             guide
+giteveryday                             guide
+gitglossary                             guide
+githooks                                guide
+gitignore                               guide
+gitmodules                              guide
+gitnamespaces                           guide
+gitrepository-layout                    guide
+gitrevisions                            guide
+gittutorial-2                           guide
+gittutorial                             guide
+gitworkflows                            guide
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 62ca8641f4..4e724a5b76 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1575,6 +1575,13 @@ _git_grep ()
 	__git_complete_refs
 }
 
+__git_all_guides=
+__git_compute_all_guides ()
+{
+	test -n "$__git_all_guides" ||
+	__git_all_guides=$(git --list-cmds=list-guide)
+}
+
 _git_help ()
 {
 	case "$cur" in
@@ -1584,11 +1591,9 @@ _git_help ()
 		;;
 	esac
 	__git_compute_all_commands
-	__gitcomp "$__git_all_commands $(__git_aliases)
-		attributes cli core-tutorial cvs-migration
-		diffcore everyday gitk glossary hooks ignore modules
-		namespaces repository-layout revisions tutorial tutorial-2
-		workflows
+	__git_compute_all_guides
+	__gitcomp "$__git_all_commands $(__git_aliases) $__git_all_guides
+		gitk
 		"
 }
 
diff --git a/help.c b/help.c
index c7df1d2338..23924dd300 100644
--- a/help.c
+++ b/help.c
@@ -39,12 +39,14 @@ static struct category_description main_categories[] = {
 	{ 0, NULL }
 };
 
-static const char *drop_prefix(const char *name)
+static const char *drop_prefix(const char *name, uint32_t category)
 {
 	const char *new_name;
 
 	if (skip_prefix(name, "git-", &new_name))
 		return new_name;
+	if (category == CAT_guide && skip_prefix(name, "git", &new_name))
+		return new_name;
 	return name;
 
 }
@@ -66,7 +68,7 @@ static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
 			continue;
 
 		cmds[nr] = *cmd;
-		cmds[nr].name = drop_prefix(cmd->name);
+		cmds[nr].name = drop_prefix(cmd->name, cmd->category);
 
 		nr++;
 	}
@@ -358,11 +360,22 @@ void list_cmds_by_category(struct string_list *list,
 	for (i = 0; i < n; i++) {
 		struct cmdname_help *cmd = command_list + i;
 
-		if (cmd->category & cat_id)
-			string_list_append(list, drop_prefix(cmd->name));
+		if (!(cmd->category & cat_id))
+			continue;
+		string_list_append(list, drop_prefix(cmd->name, cmd->category));
 	}
 }
 
+void list_common_guides_help(void)
+{
+	struct category_description catdesc[] = {
+		{ CAT_guide, N_("The common Git guides are:") },
+		{ 0, NULL }
+	};
+	print_cmd_by_category(catdesc);
+	putchar('\n');
+}
+
 void list_all_cmds_help(void)
 {
 	print_cmd_by_category(main_categories);
diff --git a/help.h b/help.h
index 40917fc38c..b2293e99be 100644
--- a/help.h
+++ b/help.h
@@ -20,6 +20,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds_help(void);
+extern void list_common_guides_help(void);
 
 extern void list_all_main_cmds(struct string_list *list);
 extern void list_all_other_cmds(struct string_list *list);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 060df24c2d..bc27df7f38 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -66,6 +66,12 @@ test_expect_success 'git help' '
 	test_i18ngrep "^   commit " help.output &&
 	test_i18ngrep "^   fetch  " help.output
 '
+test_expect_success 'git help -g' '
+	git help -g >help.output &&
+	test_i18ngrep "^   attributes " help.output &&
+	test_i18ngrep "^   everyday   " help.output &&
+	test_i18ngrep "^   tutorial   " help.output
+'
 
 test_expect_success 'generate builtin list' '
 	git --list-cmds=builtins >builtins
-- 
2.17.0.705.g3525833791
Junio C Hamano· May 8, 2018, 04:04 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v6 10/13] help: use command-list.txt for the source of guides

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 6 quoted lines
> The help command currently hard codes the list of guides and their
> summary in C. Let's move this list to command-list.txt. This lets us
> extract summary lines from Documentation/git*.txt. This also
> potentially lets us list guides in git.txt, but I'll leave that for
> now.
> ---
Nice.
Missing sign-off.
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 08/13] git: support --list-cmds=list-<category>

This allows us to select any group of commands by a category defined in command-list.txt. This is an internal/hidden option so we don't have to be picky about the category name or worried about exposing too much.

This will be used later by git-completion.bash to retrieve certain
command groups.
---
 generate-cmdlist.sh | 17 +++++++++++++++++
 git.c               |  7 +++++++
 help.c              | 23 +++++++++++++++++++++++
 help.h              |  2 ++
 4 files changed, 49 insertions(+)
Show changes to 4 files +49 −0

generate-cmdlist.sh, git.c, help.c, help.h

diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 015eef2804..bfd8ef0671 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -45,6 +45,21 @@ define_categories() {
 	test "$bit" -gt 32 && die "Urgh.. too many categories?"
 }
 
+define_category_names() {
+	echo
+	echo "/* Category names */"
+	echo "static const char *category_names[] = {"
+	bit=0
+	category_list "$1" |
+	while read cat
+	do
+		echo "	\"$cat\", /* (1UL << $bit) */"
+		bit=$(($bit+1))
+	done
+	echo "	NULL"
+	echo "};"
+}
+
 print_command_list() {
 	echo "static struct cmdname_help command_list[] = {"
 
@@ -70,4 +85,6 @@ struct cmdname_help {
 "
 define_categories "$1"
 echo
+define_category_names "$1"
+echo
 print_command_list "$1"
diff --git a/git.c b/git.c
index 3c032d01fc..67f3e20ae9 100644
--- a/git.c
+++ b/git.c
@@ -53,6 +53,13 @@ static int list_cmds(const char *spec)
 			list_all_main_cmds(&list);
 		else if (len == 6 && !strncmp(spec, "others", 6))
 			list_all_other_cmds(&list);
+		else if (len > 5 && !strncmp(spec, "list-", 5)) {
+			struct strbuf sb = STRBUF_INIT;
+
+			strbuf_add(&sb, spec + 5, len - 5);
+			list_cmds_by_category(&list, sb.buf);
+			strbuf_release(&sb);
+		}
 		else
 			die(_("unsupported command listing type '%s'"), spec);
 		spec += len;
diff --git a/help.c b/help.c
index d5ce9dfcbb..1117f7d1d1 100644
--- a/help.c
+++ b/help.c
@@ -329,6 +329,29 @@ void list_all_other_cmds(struct string_list *list)
 	clean_cmdnames(&other_cmds);
 }
 
+void list_cmds_by_category(struct string_list *list,
+			   const char *cat)
+{
+	int i, n = ARRAY_SIZE(command_list);
+	uint32_t cat_id = 0;
+
+	for (i = 0; category_names[i]; i++) {
+		if (!strcmp(cat, category_names[i])) {
+			cat_id = 1UL << i;
+			break;
+		}
+	}
+	if (!cat_id)
+		die(_("unsupported command listing type '%s'"), cat);
+
+	for (i = 0; i < n; i++) {
+		struct cmdname_help *cmd = command_list + i;
+
+		if (cmd->category & cat_id)
+			string_list_append(list, drop_prefix(cmd->name));
+	}
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 97e6c0965e..734bba32d3 100644
--- a/help.h
+++ b/help.h
@@ -21,6 +21,8 @@ static inline void mput_char(char c, unsigned int num)
 extern void list_common_cmds_help(void);
 extern void list_all_main_cmds(struct string_list *list);
 extern void list_all_other_cmds(struct string_list *list);
+extern void list_cmds_by_category(struct string_list *list,
+				  const char *category);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.705.g3525833791
Junio C Hamano· May 8, 2018, 04:00 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v6 08/13] git: support --list-cmds=list-<category>

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
Show 8 quoted lines
> This allows us to select any group of commands by a category defined
> in command-list.txt. This is an internal/hidden option so we don't
> have to be picky about the category name or worried about exposing too
> much.
>
> This will be used later by git-completion.bash to retrieve certain
> command groups.
> ---
Missing sign-off.
Show 10 quoted lines
>
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> index 015eef2804..bfd8ef0671 100755
> --- a/generate-cmdlist.sh
> +++ b/generate-cmdlist.sh
> @@ -45,6 +45,21 @@ define_categories() {
>  	test "$bit" -gt 32 && die "Urgh.. too many categories?"
>  }
>  
> +define_category_names() {
Style.
>  print_command_list() {
Ditto (but needs fixing in an earlier step).
Show 9 quoted lines
> diff --git a/git.c b/git.c
> index 3c032d01fc..67f3e20ae9 100644
> --- a/git.c
> +++ b/git.c
> @@ -53,6 +53,13 @@ static int list_cmds(const char *spec)
>  			list_all_main_cmds(&list);
>  		else if (len == 6 && !strncmp(spec, "others", 6))
>  			list_all_other_cmds(&list);
> +		else if (len > 5 && !strncmp(spec, "list-", 5)) {

Earlier I asked to have a small helper to avoid the constant length that has to go together with a constant string, e.g.

		has_prefix(spec, len, "others")
but this new one may make it a bit tricky.
	else if (has_prefix(spec, len, "others")
		...
	else if (has_prefix(spec, len, "list-") &&
		spec[strlen("list-")] != '\0' ) {
		...
does look a bit ugly.  Others may be able to help with better ideas.
Thansk.
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 11/13] command-list.txt: documentation and guide line

This is intended to help anybody who needs to update command-list.txt.
It gives a brief introduction of all attributes a command can take.
---
 command-list.txt | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
Show changes to command-list.txt +44 −0
diff --git a/command-list.txt b/command-list.txt
index 99ddc231c1..9c70c69193 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -1,3 +1,47 @@
+# Command classification list
+# ---------------------------
+# All supported commands, builtin or external, must be described in
+# here. This info is used to list commands in various places. Each
+# command is on one line followed by one or more attributes.
+#
+# The first attribute group is mandatory and indicates the command
+# type. This group includes:
+#
+#   mainporcelain
+#   ancillarymanipulators
+#   ancillaryinterrogators
+#   foreignscminterface
+#   plumbingmanipulators
+#   plumbinginterrogators
+#   synchingrepositories
+#   synchelpers
+#   purehelpers
+#
+# The type names are self explanatory. But if you want to see what
+# command belongs to what group to get a better picture, have a look
+# at "git" man page, "GIT COMMANDS" section.
+#
+# Commands of type mainporcelain can also optionally have one of these
+# attributes:
+#
+#   init
+#   worktree
+#   info
+#   history
+#   remote
+#
+# These commands are considered "common" and will show up in "git
+# help" output in groups. Uncommon porcelain commands must not
+# specify any of these attributes.
+#
+# "complete" attribute is used to mark that the command should be
+# completable by git-completion.bash. Note that by default,
+# mainporcelain commands are completable so you don't need this
+# attribute.
+#
+# While not true commands, guides are also specified here, which can
+# only have "guide" attribute and nothing else.
+#
 ### command list (do not change this line, also do not change alignment)
 # command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
-- 
2.17.0.705.g3525833791
Philip Oakley· May 12, 2018, 17:50 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v6 11/13] command-list.txt: documentation and guide line

Hi Duy,
From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com> : Monday, May 07, 2018
Show 54 quoted lines
> This is intended to help anybody who needs to update command-list.txt.
> It gives a brief introduction of all attributes a command can take.
> ---
> command-list.txt | 44 ++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 44 insertions(+)
>
> diff --git a/command-list.txt b/command-list.txt
> index 99ddc231c1..9c70c69193 100644
> --- a/command-list.txt
> +++ b/command-list.txt
> @@ -1,3 +1,47 @@
> +# Command classification list
> +# ---------------------------
> +# All supported commands, builtin or external, must be described in
> +# here. This info is used to list commands in various places. Each
> +# command is on one line followed by one or more attributes.
> +#
> +# The first attribute group is mandatory and indicates the command
> +# type. This group includes:
> +#
> +#   mainporcelain
> +#   ancillarymanipulators
> +#   ancillaryinterrogators
> +#   foreignscminterface
> +#   plumbingmanipulators
> +#   plumbinginterrogators
> +#   synchingrepositories
> +#   synchelpers
> +#   purehelpers
> +#
> +# The type names are self explanatory. But if you want to see what
> +# command belongs to what group to get a better picture, have a look
> +# at "git" man page, "GIT COMMANDS" section.
> +#
> +# Commands of type mainporcelain can also optionally have one of these
> +# attributes:
> +#
> +#   init
> +#   worktree
> +#   info
> +#   history
> +#   remote
> +#
> +# These commands are considered "common" and will show up in "git
> +# help" output in groups. Uncommon porcelain commands must not
> +# specify any of these attributes.
> +#
> +# "complete" attribute is used to mark that the command should be
> +# completable by git-completion.bash. Note that by default,
> +# mainporcelain commands are completable so you don't need this
> +# attribute.
> +#
> +# While not true commands, guides are also specified here, which can
> +# only have "guide" attribute and nothing else.

While the file is called ~ "Command List", the list is here as a support to the Help function, and ultimately to the user's reading of the man pages, including the man(5/7) guides, so I'd view the man page guides as first class citizens.

Perhaps: # As part of the Git man page list, the man(5/7) guides are also specified # here, which can only have "guide" attribute and nothing else.

-- Philip

Show 8 quoted lines
> +#
> ### command list (do not change this line, also do not change alignment)
> # command name                          category [category] [category]
> git-add                                 mainporcelain           worktree
> -- 
> 2.17.0.705.g3525833791
>
>
Nguyễn Thái Ngọc Duy· May 7, 2018, 17:52 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v6 03/13] help: use command-list.h for common command list

The previous commit added code generation for all_cmd_desc[] which includes almost everything we need to generate common command list. Convert help code to use that array instead and drop common_cmds[] array.

The description of each common command group is removed from
command-list.txt. This keeps this file format simpler. common-cmds.h
will not be generated correctly after this change due to the
command-list.txt format change. But it does not matter and
common-cmds.h will be removed.
---
 Makefile            |   4 +-
 command-list.txt    |  10 ---
 generate-cmdlist.sh |   4 +-
 help.c              | 145 +++++++++++++++++++++++++++++++++-----------
 t/t0012-help.sh     |   9 +++
 5 files changed, 122 insertions(+), 50 deletions(-)
Show changes to 5 files +122 −50

Makefile, command-list.txt, generate-cmdlist.sh, help.c, t/t0012-help.sh

diff --git a/Makefile b/Makefile
index 2a8913ea21..5c58b0b692 100644
--- a/Makefile
+++ b/Makefile
@@ -1914,9 +1914,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
-help.sp help.s help.o: common-cmds.h
+help.sp help.s help.o: common-cmds.h command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h command-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
diff --git a/command-list.txt b/command-list.txt
index 786536aba0..3bd23201a6 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -1,13 +1,3 @@
-# common commands are grouped by themes
-# these groups are output by 'git help' in the order declared here.
-# map each common command in the command list to one of these groups.
-### common groups (do not change this line)
-init         start a working area (see also: git help tutorial)
-worktree     work on the current change (see also: git help everyday)
-info         examine the history and state (see also: git help revisions)
-history      grow, mark and tweak your common history
-remote       collaborate (see also: git help workflows)
-
 ### command list (do not change this line, also do not change alignment)
 # command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index c9fd524760..93de8e8f59 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -6,7 +6,7 @@ die () {
 }
 
 command_list () {
-	sed '1,/^### command list/d;/^#/d' "$1"
+	grep -v '^#' "$1"
 }
 
 get_categories() {
@@ -65,7 +65,7 @@ echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
 	const char *name;
 	const char *help;
-	uint32_t group;
+	uint32_t category;
 };
 "
 if [ -z "$2" ]
diff --git a/help.c b/help.c
index 60071a9bea..2d6a3157f8 100644
--- a/help.c
+++ b/help.c
@@ -5,13 +5,114 @@
 #include "run-command.h"
 #include "levenshtein.h"
 #include "help.h"
-#include "common-cmds.h"
+#include "command-list.h"
 #include "string-list.h"
 #include "column.h"
 #include "version.h"
 #include "refs.h"
 #include "parse-options.h"
 
+struct category_description {
+	uint32_t category;
+	const char *desc;
+};
+static uint32_t common_mask =
+	CAT_init | CAT_worktree | CAT_info |
+	CAT_history | CAT_remote;
+static struct category_description common_categories[] = {
+	{ CAT_init, N_("start a working area (see also: git help tutorial)") },
+	{ CAT_worktree, N_("work on the current change (see also: git help everyday)") },
+	{ CAT_info, N_("examine the history and state (see also: git help revisions)") },
+	{ CAT_history, N_("grow, mark and tweak your common history") },
+	{ CAT_remote, N_("collaborate (see also: git help workflows)") },
+	{ 0, NULL }
+};
+
+static const char *drop_prefix(const char *name)
+{
+	const char *new_name;
+
+	if (skip_prefix(name, "git-", &new_name))
+		return new_name;
+	return name;
+
+}
+
+static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
+{
+	int i, nr = 0;
+	struct cmdname_help *cmds;
+
+	if (ARRAY_SIZE(command_list) == 0)
+		BUG("empty command_list[] is a sign of broken generate-cmdlist.sh");
+
+	ALLOC_ARRAY(cmds, ARRAY_SIZE(command_list) + 1);
+
+	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
+		const struct cmdname_help *cmd = command_list + i;
+
+		if (!(cmd->category & mask))
+			continue;
+
+		cmds[nr] = *cmd;
+		cmds[nr].name = drop_prefix(cmd->name);
+
+		nr++;
+	}
+	cmds[nr].name = NULL;
+	*p_cmds = cmds;
+}
+
+static void print_command_list(const struct cmdname_help *cmds,
+			       uint32_t mask, int longest)
+{
+	int i;
+
+	for (i = 0; cmds[i].name; i++) {
+		if (cmds[i].category & mask) {
+			printf("   %s   ", cmds[i].name);
+			mput_char(' ', longest - strlen(cmds[i].name));
+			puts(_(cmds[i].help));
+		}
+	}
+}
+
+static int cmd_name_cmp(const void *elem1, const void *elem2)
+{
+	const struct cmdname_help *e1 = elem1;
+	const struct cmdname_help *e2 = elem2;
+
+	return strcmp(e1->name, e2->name);
+}
+
+static void print_cmd_by_category(const struct category_description *catdesc)
+{
+	struct cmdname_help *cmds;
+	int longest = 0;
+	int i, nr = 0;
+	uint32_t mask = 0;
+
+	for (i = 0; catdesc[i].desc; i++)
+		mask |= catdesc[i].category;
+
+	extract_cmds(&cmds, mask);
+
+	for (i = 0; cmds[i].name; i++, nr++) {
+		if (longest < strlen(cmds[i].name))
+			longest = strlen(cmds[i].name);
+	}
+	QSORT(cmds, nr, cmd_name_cmp);
+
+	for (i = 0; catdesc[i].desc; i++) {
+		uint32_t mask = catdesc[i].category;
+		const char *desc = catdesc[i].desc;
+
+		printf("\n%s\n", _(desc));
+		print_command_list(cmds, mask, longest);
+	}
+	free(cmds);
+}
+
 void add_cmdname(struct cmdnames *cmds, const char *name, int len)
 {
 	struct cmdname *ent;
@@ -190,42 +291,10 @@ void list_commands(unsigned int colopts,
 	}
 }
 
-static int cmd_group_cmp(const void *elem1, const void *elem2)
-{
-	const struct cmdname_help *e1 = elem1;
-	const struct cmdname_help *e2 = elem2;
-
-	if (e1->group < e2->group)
-		return -1;
-	if (e1->group > e2->group)
-		return 1;
-	return strcmp(e1->name, e2->name);
-}
-
 void list_common_cmds_help(void)
 {
-	int i, longest = 0;
-	int current_grp = -1;
-
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
-		if (longest < strlen(common_cmds[i].name))
-			longest = strlen(common_cmds[i].name);
-	}
-
-	QSORT(common_cmds, ARRAY_SIZE(common_cmds), cmd_group_cmp);
-
 	puts(_("These are common Git commands used in various situations:"));
-
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
-		if (common_cmds[i].group != current_grp) {
-			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
-			current_grp = common_cmds[i].group;
-		}
-
-		printf("   %s   ", common_cmds[i].name);
-		mput_char(' ', longest - strlen(common_cmds[i].name));
-		puts(_(common_cmds[i].help));
-	}
+	print_cmd_by_category(common_categories);
 }
 
 int is_in_cmdlist(struct cmdnames *c, const char *s)
@@ -285,6 +354,7 @@ const char *help_unknown_cmd(const char *cmd)
 {
 	int i, n, best_similarity = 0;
 	struct cmdnames main_cmds, other_cmds;
+	struct cmdname_help *common_cmds;
 
 	memset(&main_cmds, 0, sizeof(main_cmds));
 	memset(&other_cmds, 0, sizeof(other_cmds));
@@ -299,6 +369,8 @@ const char *help_unknown_cmd(const char *cmd)
 	QSORT(main_cmds.names, main_cmds.cnt, cmdname_compare);
 	uniq(&main_cmds);
 
+	extract_cmds(&common_cmds, common_mask);
+
 	/* This abuses cmdname->len for levenshtein distance */
 	for (i = 0, n = 0; i < main_cmds.cnt; i++) {
 		int cmp = 0; /* avoid compiler stupidity */
@@ -313,10 +385,10 @@ const char *help_unknown_cmd(const char *cmd)
 			die(_(bad_interpreter_advice), cmd, cmd);
 
 		/* Does the candidate appear in common_cmds list? */
-		while (n < ARRAY_SIZE(common_cmds) &&
+		while (common_cmds[n].name &&
 		       (cmp = strcmp(common_cmds[n].name, candidate)) < 0)
 			n++;
-		if ((n < ARRAY_SIZE(common_cmds)) && !cmp) {
+		if (common_cmds[n].name && !cmp) {
 			/* Yes, this is one of the common commands */
 			n++; /* use the entry from common_cmds[] */
 			if (starts_with(candidate, cmd)) {
@@ -329,6 +401,7 @@ const char *help_unknown_cmd(const char *cmd)
 		main_cmds.names[i]->len =
 			levenshtein(cmd, candidate, 0, 2, 1, 3) + 1;
 	}
+	FREE_AND_NULL(common_cmds);
 
 	QSORT(main_cmds.names, main_cmds.cnt, levenshtein_compare);
 
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 487b92a5de..c096f33505 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -49,6 +49,15 @@ test_expect_success "--help does not work for guides" "
 	test_i18ncmp expect actual
 "
 
+test_expect_success 'git help' '
+	git help >help.output &&
+	test_i18ngrep "^   clone  " help.output &&
+	test_i18ngrep "^   add    " help.output &&
+	test_i18ngrep "^   log    " help.output &&
+	test_i18ngrep "^   commit " help.output &&
+	test_i18ngrep "^   fetch  " help.output
+'
+
 test_expect_success 'generate builtin list' '
 	git --list-builtins >builtins
 '
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 00/13] Keep all info in command-list.txt in git binary

v7 is mostly code cleanup plus one more change:

git-completion.bash now always checks completion.commands config key. This makes it much more convenient to customize the command complete list. You only have to fall back to setting $GIT_COMPLETION_CMD_GROUPS when you have very special needs.

Interdiff
Show changes to 4 files +29 −21

Documentation/config.txt, contrib/completion/git-completion.bash, generate-cmdlist.sh, git.c

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 2659153cb3..91f7eaed7b 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1343,6 +1343,16 @@ credential.<url>.*::
 credentialCache.ignoreSIGHUP::
 	Tell git-credential-cache--daemon to ignore SIGHUP, instead of quitting.
 
+completion.commands::
+	This is only used by git-completion.bash to add or remove
+	commands from the complete list. Normally only porcelain
+	commands and a few select others are in the complete list. You
+	can add more commands, separated by space, in this
+	variable. Prefixing a command with '-' will remove it from
+	the existing list.
++
+This variable should not be per-repository.
+
 include::diff-config.txt[]
 
 difftool.<tool>.path::
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 0fd29803d5..f7ca9a4d4f 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -50,10 +50,10 @@
 #
 #         GIT_COMPLETION_CMD_GROUPS=main,others
 #
-#     Or you could go with defaults add some extra commands specified
-#     in the configuration variable completion.commands [1] with
+#     Or you could go with main porcelain only and extra commands in
+#     the configuration variable completion.commands with
 #
-#         GIT_COMPLETION_CMD_GROUPS=mainporcelain,others,list-complete,config
+#         GIT_COMPLETION_CMD_GROUPS=mainporcelain,config
 #
 #     Or go completely custom group with
 #
@@ -61,15 +61,6 @@
 #
 #     Or you could even play with other command categories found in
 #     command-list.txt.
-#
-#     [1] Note that completion.commands should not be per-repository
-#         since the command list is generated once and cached.
-#
-#         completion.commands could be used to exclude commands as
-#         well.  If a command in this list begins with '-', then it
-#         will be excluded from the list of commands gathered by the
-#         groups specified before "config" in
-#         $GIT_COMPLETION_CMD_GROUPS.
 
 case "$COMP_WORDBREAKS" in
 *:*) : great ;;
@@ -876,7 +867,7 @@ __git_commands () {
 		then
 			git --list-cmds="$GIT_COMPLETION_CMD_GROUPS"
 		else
-			git --list-cmds=list-mainporcelain,others,list-complete
+			git --list-cmds=list-mainporcelain,others,list-complete,config
 		fi
 		;;
 	all)
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index bfd8ef0671..8d6d8b45ce 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -9,7 +9,7 @@ command_list () {
 	grep -v '^#' "$1"
 }
 
-get_categories() {
+get_categories () {
 	tr ' ' '\n'|
 	grep -v '^$' |
 	sort |
@@ -32,7 +32,7 @@ get_synopsis () {
 		}' "Documentation/$1.txt"
 }
 
-define_categories() {
+define_categories () {
 	echo
 	echo "/* Command categories */"
 	bit=0
@@ -45,7 +45,7 @@ define_categories() {
 	test "$bit" -gt 32 && die "Urgh.. too many categories?"
 }
 
-define_category_names() {
+define_category_names () {
 	echo
 	echo "/* Category names */"
 	echo "static const char *category_names[] = {"
@@ -60,7 +60,7 @@ define_category_names() {
 	echo "};"
 }
 
-print_command_list() {
+print_command_list () {
 	echo "static struct cmdname_help command_list[] = {"
 
 	command_list "$1" |
diff --git a/git.c b/git.c
index fd08911e11..ea4feedd0b 100644
--- a/git.c
+++ b/git.c
@@ -38,6 +38,13 @@ static int use_pager = -1;
 
 static void list_builtins(struct string_list *list, unsigned int exclude_option);
 
+static int match_token(const char *spec, int len, const char *token)
+{
+	int token_len = strlen(token);
+
+	return len == token_len && !strncmp(spec, token, token_len);
+}
+
 static int list_cmds(const char *spec)
 {
 	struct string_list list = STRING_LIST_INIT_DUP;
@@ -47,13 +54,13 @@ static int list_cmds(const char *spec)
 		const char *sep = strchrnul(spec, ',');
 		int len = sep - spec;
 
-		if (len == 8 && !strncmp(spec, "builtins", 8))
+		if (match_token(spec, len, "builtins"))
 			list_builtins(&list, 0);
-		else if (len == 4 && !strncmp(spec, "main", 4))
+		else if (match_token(spec, len, "main"))
 			list_all_main_cmds(&list);
-		else if (len == 6 && !strncmp(spec, "others", 6))
+		else if (match_token(spec, len, "others"))
 			list_all_other_cmds(&list);
-		else if (len == 6 && !strncmp(spec, "config", 6))
+		else if (match_token(spec, len, "config"))
 			list_cmds_by_config(&list);
 		else if (len > 5 && !strncmp(spec, "list-", 5)) {
 			struct strbuf sb = STRBUF_INIT;

Nguyễn Thái Ngọc Duy (13):
  generate-cmds.sh: factor out synopsis extract code
  generate-cmds.sh: export all commands to command-list.h
  help: use command-list.h for common command list
  Remove common-cmds.h
  git.c: convert --list-* to --list-cmds=*
  git --list-cmds: collect command list in a string_list
  completion: implement and use --list-cmds=main,others
  git: support --list-cmds=list-<category>
  help: add "-a --verbose" to list all commands with synopsis
  help: use command-list.txt for the source of guides
  command-list.txt: documentation and guide line
  completion: let git provide the completable command list
  completion: allow to customize the completable command list

 .gitignore                             |   2 +-
 Documentation/config.txt               |  10 +
 Documentation/git-help.txt             |   4 +-
 Documentation/gitattributes.txt        |   2 +-
 Documentation/gitmodules.txt           |   2 +-
 Documentation/gitrevisions.txt         |   2 +-
 Makefile                               |  16 +-
 builtin/help.c                         |  39 +---
 command-list.txt                       | 111 ++++++++---
 contrib/completion/git-completion.bash | 160 +++++++---------
 generate-cmdlist.sh                    | 126 ++++++++-----
 git.c                                  |  68 ++++++-
 help.c                                 | 244 ++++++++++++++++++++++---
 help.h                                 |  10 +
 t/t0012-help.sh                        |  26 ++-
 t/t9902-completion.sh                  |   5 +-
 16 files changed, 573 insertions(+), 254 deletions(-)
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 01/13] generate-cmds.sh: factor out synopsis extract code

This makes it easier to reuse the same code in another place (very soon).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 generate-cmdlist.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
Show changes to generate-cmdlist.sh +11 −7
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index eeea4b67ea..31b6d886cb 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,15 @@
 #!/bin/sh
 
+get_synopsis () {
+	sed -n '
+		/^NAME/,/'"$1"'/H
+		${
+			x
+			s/.*'"$1"' - \(.*\)/N_("\1")/
+			p
+		}' "Documentation/$1.txt"
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
 	char name[16];
@@ -39,12 +49,6 @@ sort |
 while read cmd tags
 do
 	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
-	sed -n '
-		/^NAME/,/git-'"$cmd"'/H
-		${
-			x
-			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
-			p
-		}' "Documentation/git-$cmd.txt"
+	echo "	{\"$cmd\", $(get_synopsis git-$cmd), $tag},"
 done
 echo "};"
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 03/13] help: use command-list.h for common command list

The previous commit added code generation for all_cmd_desc[] which includes almost everything we need to generate common command list. Convert help code to use that array instead and drop common_cmds[] array.

The description of each common command group is removed from command-list.txt. This keeps this file format simpler. common-cmds.h will not be generated correctly after this change due to the command-list.txt format change. But it does not matter and common-cmds.h will be removed.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Makefile            |   4 +-
 command-list.txt    |  10 ---
 generate-cmdlist.sh |   4 +-
 help.c              | 145 +++++++++++++++++++++++++++++++++-----------
 t/t0012-help.sh     |   9 +++
 5 files changed, 122 insertions(+), 50 deletions(-)
Show changes to 5 files +122 −50

Makefile, command-list.txt, generate-cmdlist.sh, help.c, t/t0012-help.sh

diff --git a/Makefile b/Makefile
index 2a8913ea21..5c58b0b692 100644
--- a/Makefile
+++ b/Makefile
@@ -1914,9 +1914,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
-help.sp help.s help.o: common-cmds.h
+help.sp help.s help.o: common-cmds.h command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h command-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
diff --git a/command-list.txt b/command-list.txt
index 786536aba0..3bd23201a6 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -1,13 +1,3 @@
-# common commands are grouped by themes
-# these groups are output by 'git help' in the order declared here.
-# map each common command in the command list to one of these groups.
-### common groups (do not change this line)
-init         start a working area (see also: git help tutorial)
-worktree     work on the current change (see also: git help everyday)
-info         examine the history and state (see also: git help revisions)
-history      grow, mark and tweak your common history
-remote       collaborate (see also: git help workflows)
-
 ### command list (do not change this line, also do not change alignment)
 # command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 870d3b626a..9eb22c4ef1 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -6,7 +6,7 @@ die () {
 }
 
 command_list () {
-	sed '1,/^### command list/d;/^#/d' "$1"
+	grep -v '^#' "$1"
 }
 
 get_categories () {
@@ -65,7 +65,7 @@ echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
 	const char *name;
 	const char *help;
-	uint32_t group;
+	uint32_t category;
 };
 "
 if test -z "$2"
diff --git a/help.c b/help.c
index 60071a9bea..2d6a3157f8 100644
--- a/help.c
+++ b/help.c
@@ -5,13 +5,114 @@
 #include "run-command.h"
 #include "levenshtein.h"
 #include "help.h"
-#include "common-cmds.h"
+#include "command-list.h"
 #include "string-list.h"
 #include "column.h"
 #include "version.h"
 #include "refs.h"
 #include "parse-options.h"
 
+struct category_description {
+	uint32_t category;
+	const char *desc;
+};
+static uint32_t common_mask =
+	CAT_init | CAT_worktree | CAT_info |
+	CAT_history | CAT_remote;
+static struct category_description common_categories[] = {
+	{ CAT_init, N_("start a working area (see also: git help tutorial)") },
+	{ CAT_worktree, N_("work on the current change (see also: git help everyday)") },
+	{ CAT_info, N_("examine the history and state (see also: git help revisions)") },
+	{ CAT_history, N_("grow, mark and tweak your common history") },
+	{ CAT_remote, N_("collaborate (see also: git help workflows)") },
+	{ 0, NULL }
+};
+
+static const char *drop_prefix(const char *name)
+{
+	const char *new_name;
+
+	if (skip_prefix(name, "git-", &new_name))
+		return new_name;
+	return name;
+
+}
+
+static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
+{
+	int i, nr = 0;
+	struct cmdname_help *cmds;
+
+	if (ARRAY_SIZE(command_list) == 0)
+		BUG("empty command_list[] is a sign of broken generate-cmdlist.sh");
+
+	ALLOC_ARRAY(cmds, ARRAY_SIZE(command_list) + 1);
+
+	for (i = 0; i < ARRAY_SIZE(command_list); i++) {
+		const struct cmdname_help *cmd = command_list + i;
+
+		if (!(cmd->category & mask))
+			continue;
+
+		cmds[nr] = *cmd;
+		cmds[nr].name = drop_prefix(cmd->name);
+
+		nr++;
+	}
+	cmds[nr].name = NULL;
+	*p_cmds = cmds;
+}
+
+static void print_command_list(const struct cmdname_help *cmds,
+			       uint32_t mask, int longest)
+{
+	int i;
+
+	for (i = 0; cmds[i].name; i++) {
+		if (cmds[i].category & mask) {
+			printf("   %s   ", cmds[i].name);
+			mput_char(' ', longest - strlen(cmds[i].name));
+			puts(_(cmds[i].help));
+		}
+	}
+}
+
+static int cmd_name_cmp(const void *elem1, const void *elem2)
+{
+	const struct cmdname_help *e1 = elem1;
+	const struct cmdname_help *e2 = elem2;
+
+	return strcmp(e1->name, e2->name);
+}
+
+static void print_cmd_by_category(const struct category_description *catdesc)
+{
+	struct cmdname_help *cmds;
+	int longest = 0;
+	int i, nr = 0;
+	uint32_t mask = 0;
+
+	for (i = 0; catdesc[i].desc; i++)
+		mask |= catdesc[i].category;
+
+	extract_cmds(&cmds, mask);
+
+	for (i = 0; cmds[i].name; i++, nr++) {
+		if (longest < strlen(cmds[i].name))
+			longest = strlen(cmds[i].name);
+	}
+	QSORT(cmds, nr, cmd_name_cmp);
+
+	for (i = 0; catdesc[i].desc; i++) {
+		uint32_t mask = catdesc[i].category;
+		const char *desc = catdesc[i].desc;
+
+		printf("\n%s\n", _(desc));
+		print_command_list(cmds, mask, longest);
+	}
+	free(cmds);
+}
+
 void add_cmdname(struct cmdnames *cmds, const char *name, int len)
 {
 	struct cmdname *ent;
@@ -190,42 +291,10 @@ void list_commands(unsigned int colopts,
 	}
 }
 
-static int cmd_group_cmp(const void *elem1, const void *elem2)
-{
-	const struct cmdname_help *e1 = elem1;
-	const struct cmdname_help *e2 = elem2;
-
-	if (e1->group < e2->group)
-		return -1;
-	if (e1->group > e2->group)
-		return 1;
-	return strcmp(e1->name, e2->name);
-}
-
 void list_common_cmds_help(void)
 {
-	int i, longest = 0;
-	int current_grp = -1;
-
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
-		if (longest < strlen(common_cmds[i].name))
-			longest = strlen(common_cmds[i].name);
-	}
-
-	QSORT(common_cmds, ARRAY_SIZE(common_cmds), cmd_group_cmp);
-
 	puts(_("These are common Git commands used in various situations:"));
-
-	for (i = 0; i < ARRAY_SIZE(common_cmds); i++) {
-		if (common_cmds[i].group != current_grp) {
-			printf("\n%s\n", _(common_cmd_groups[common_cmds[i].group]));
-			current_grp = common_cmds[i].group;
-		}
-
-		printf("   %s   ", common_cmds[i].name);
-		mput_char(' ', longest - strlen(common_cmds[i].name));
-		puts(_(common_cmds[i].help));
-	}
+	print_cmd_by_category(common_categories);
 }
 
 int is_in_cmdlist(struct cmdnames *c, const char *s)
@@ -285,6 +354,7 @@ const char *help_unknown_cmd(const char *cmd)
 {
 	int i, n, best_similarity = 0;
 	struct cmdnames main_cmds, other_cmds;
+	struct cmdname_help *common_cmds;
 
 	memset(&main_cmds, 0, sizeof(main_cmds));
 	memset(&other_cmds, 0, sizeof(other_cmds));
@@ -299,6 +369,8 @@ const char *help_unknown_cmd(const char *cmd)
 	QSORT(main_cmds.names, main_cmds.cnt, cmdname_compare);
 	uniq(&main_cmds);
 
+	extract_cmds(&common_cmds, common_mask);
+
 	/* This abuses cmdname->len for levenshtein distance */
 	for (i = 0, n = 0; i < main_cmds.cnt; i++) {
 		int cmp = 0; /* avoid compiler stupidity */
@@ -313,10 +385,10 @@ const char *help_unknown_cmd(const char *cmd)
 			die(_(bad_interpreter_advice), cmd, cmd);
 
 		/* Does the candidate appear in common_cmds list? */
-		while (n < ARRAY_SIZE(common_cmds) &&
+		while (common_cmds[n].name &&
 		       (cmp = strcmp(common_cmds[n].name, candidate)) < 0)
 			n++;
-		if ((n < ARRAY_SIZE(common_cmds)) && !cmp) {
+		if (common_cmds[n].name && !cmp) {
 			/* Yes, this is one of the common commands */
 			n++; /* use the entry from common_cmds[] */
 			if (starts_with(candidate, cmd)) {
@@ -329,6 +401,7 @@ const char *help_unknown_cmd(const char *cmd)
 		main_cmds.names[i]->len =
 			levenshtein(cmd, candidate, 0, 2, 1, 3) + 1;
 	}
+	FREE_AND_NULL(common_cmds);
 
 	QSORT(main_cmds.names, main_cmds.cnt, levenshtein_compare);
 
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 487b92a5de..c096f33505 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -49,6 +49,15 @@ test_expect_success "--help does not work for guides" "
 	test_i18ncmp expect actual
 "
 
+test_expect_success 'git help' '
+	git help >help.output &&
+	test_i18ngrep "^   clone  " help.output &&
+	test_i18ngrep "^   add    " help.output &&
+	test_i18ngrep "^   log    " help.output &&
+	test_i18ngrep "^   commit " help.output &&
+	test_i18ngrep "^   fetch  " help.output
+'
+
 test_expect_success 'generate builtin list' '
 	git --list-builtins >builtins
 '
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 02/13] generate-cmds.sh: export all commands to command-list.h

The current generate-cmds.sh generates just enough to print "git help" output. That is, it only extracts help text for common commands.

The script is now updated to extract help text for all commands and keep command classification a new file, command-list.h. This will be useful later:

- "git help -a" could print a short summary of all commands instead of
  just the common ones.
- "git" could produce a list of commands of one or more category. One
  of its use is to reduce another command classification embedded in
  git-completion.bash.

The new file can be generated but is not used anywhere yet. The plan is we migrate away from common-cmds.h. Then we can kill off common-cmds.h build rules and generation code (and also delete duplicate content in command-list.h which we keep for now to not mess generate-cmds.sh up too much).

PS. The new fixed column requirement on command-list.txt is technically not needed. But it helps simplify the code a bit at this stage. We could lift this restriction later if we want to.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 .gitignore          |  1 +
 Makefile            | 13 ++++++---
 command-list.txt    |  4 +--
 generate-cmdlist.sh | 67 ++++++++++++++++++++++++++++++++++++++++++---
 4 files changed, 75 insertions(+), 10 deletions(-)
Show changes to 4 files +75 −10

.gitignore, Makefile, command-list.txt, generate-cmdlist.sh

diff --git a/.gitignore b/.gitignore
index 833ef3b0b7..d4c3914167 100644
--- a/.gitignore
+++ b/.gitignore
@@ -180,6 +180,7 @@
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
 /common-cmds.h
+/command-list.h
 *.tar.gz
 *.dsc
 *.deb
diff --git a/Makefile b/Makefile
index f181687250..2a8913ea21 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h
+GENERATED_H += common-cmds.h command-list.h
 
 LIB_H = $(shell $(FIND) . \
 	-name .git -prune -o \
@@ -1938,6 +1938,11 @@ $(BUILT_INS): git$X
 common-cmds.h: generate-cmdlist.sh command-list.txt
 
 common-cmds.h: $(wildcard Documentation/git-*.txt)
+	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON >$@+ && mv $@+ $@
+
+command-list.h: generate-cmdlist.sh command-list.txt
+
+command-list.h: $(wildcard Documentation/git-*.txt)
 	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
@@ -2148,7 +2153,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h
+# Dependencies on automatically generated headers such as common-cmds.h or command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2527,7 +2532,7 @@ sparse: $(SP_OBJ)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h
+check: common-cmds.h command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -2775,7 +2780,7 @@ clean: profile-clean coverage-clean
 	$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/command-list.txt b/command-list.txt
index a1fad28fd8..786536aba0 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -8,8 +8,8 @@ info         examine the history and state (see also: git help revisions)
 history      grow, mark and tweak your common history
 remote       collaborate (see also: git help workflows)
 
-### command list (do not change this line)
-# command name                          category [deprecated] [common]
+### command list (do not change this line, also do not change alignment)
+# command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 31b6d886cb..870d3b626a 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -1,5 +1,27 @@
 #!/bin/sh
 
+die () {
+	echo "$@" >&2
+	exit 1
+}
+
+command_list () {
+	sed '1,/^### command list/d;/^#/d' "$1"
+}
+
+get_categories () {
+	tr ' ' '\n'|
+	grep -v '^$' |
+	sort |
+	uniq
+}
+
+category_list () {
+	command_list "$1" |
+	cut -c 40- |
+	get_categories
+}
+
 get_synopsis () {
 	sed -n '
 		/^NAME/,/'"$1"'/H
@@ -10,14 +32,51 @@ get_synopsis () {
 		}' "Documentation/$1.txt"
 }
 
+define_categories () {
+	echo
+	echo "/* Command categories */"
+	bit=0
+	category_list "$1" |
+	while read cat
+	do
+		echo "#define CAT_$cat (1UL << $bit)"
+		bit=$(($bit+1))
+	done
+	test "$bit" -gt 32 && die "Urgh.. too many categories?"
+}
+
+print_command_list () {
+	echo "static struct cmdname_help command_list[] = {"
+
+	command_list "$1" |
+	while read cmd rest
+	do
+		printf "	{ \"$cmd\", $(get_synopsis $cmd), 0"
+		for cat in $(echo "$rest" | get_categories)
+		do
+			printf " | CAT_$cat"
+		done
+		echo " },"
+	done
+	echo "};"
+}
+
 echo "/* Automatically generated by generate-cmdlist.sh */
 struct cmdname_help {
-	char name[16];
-	char help[80];
-	unsigned char group;
+	const char *name;
+	const char *help;
+	uint32_t group;
 };
+"
+if test -z "$2"
+then
+	define_categories "$1"
+	echo
+	print_command_list "$1"
+	exit 0
+fi
 
-static const char *common_cmd_groups[] = {"
+echo "static const char *common_cmd_groups[] = {"
 
 grps=grps$$.tmp
 match=match$$.tmp
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 04/13] Remove common-cmds.h

After the last patch, common-cmds.h is no longer used (and it was actually broken). Remove all related code. command-list.h will take its place from now on.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 .gitignore          |  1 -
 Makefile            | 17 ++++++-----------
 generate-cmdlist.sh | 46 +++------------------------------------------
 3 files changed, 9 insertions(+), 55 deletions(-)
Show changes to 3 files +9 −55

.gitignore, Makefile, generate-cmdlist.sh

diff --git a/.gitignore b/.gitignore
index d4c3914167..0836083992 100644
--- a/.gitignore
+++ b/.gitignore
@@ -179,7 +179,6 @@
 /gitweb/gitweb.cgi
 /gitweb/static/gitweb.js
 /gitweb/static/gitweb.min.*
-/common-cmds.h
 /command-list.h
 *.tar.gz
 *.dsc
diff --git a/Makefile b/Makefile
index 5c58b0b692..a60a78ee67 100644
--- a/Makefile
+++ b/Makefile
@@ -757,7 +757,7 @@ LIB_FILE = libgit.a
 XDIFF_LIB = xdiff/lib.a
 VCSSVN_LIB = vcs-svn/lib.a
 
-GENERATED_H += common-cmds.h command-list.h
+GENERATED_H += command-list.h
 
 LIB_H = $(shell $(FIND) . \
 	-name .git -prune -o \
@@ -1914,9 +1914,9 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
 	$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) \
 		$(filter %.o,$^) $(LIBS)
 
-help.sp help.s help.o: common-cmds.h command-list.h
+help.sp help.s help.o: command-list.h
 
-builtin/help.sp builtin/help.s builtin/help.o: common-cmds.h command-list.h GIT-PREFIX
+builtin/help.sp builtin/help.s builtin/help.o: command-list.h GIT-PREFIX
 builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
 	'-DGIT_HTML_PATH="$(htmldir_relative_SQ)"' \
 	'-DGIT_MAN_PATH="$(mandir_relative_SQ)"' \
@@ -1935,11 +1935,6 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
-common-cmds.h: generate-cmdlist.sh command-list.txt
-
-common-cmds.h: $(wildcard Documentation/git-*.txt)
-	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt COMMON >$@+ && mv $@+ $@
-
 command-list.h: generate-cmdlist.sh command-list.txt
 
 command-list.h: $(wildcard Documentation/git-*.txt)
@@ -2153,7 +2148,7 @@ else
 # Dependencies on header files, for platforms that do not support
 # the gcc -MMD option.
 #
-# Dependencies on automatically generated headers such as common-cmds.h or command-list.h
+# Dependencies on automatically generated headers such as command-list.h
 # should _not_ be included here, since they are necessary even when
 # building an object for the first time.
 
@@ -2532,7 +2527,7 @@ sparse: $(SP_OBJ)
 style:
 	git clang-format --style file --diff --extensions c,h
 
-check: common-cmds.h command-list.h
+check: command-list.h
 	@if sparse; \
 	then \
 		echo >&2 "Use 'make sparse' instead"; \
@@ -2780,7 +2775,7 @@ clean: profile-clean coverage-clean
 	$(RM) $(TEST_PROGRAMS) $(NO_INSTALL)
 	$(RM) -r bin-wrappers $(dep_dirs)
 	$(RM) -r po/build/
-	$(RM) *.pyc *.pyo */*.pyc */*.pyo common-cmds.h command-list.h $(ETAGS_TARGET) tags cscope*
+	$(RM) *.pyc *.pyo */*.pyc */*.pyo command-list.h $(ETAGS_TARGET) tags cscope*
 	$(RM) -r $(GIT_TARNAME) .doc-tmp-dir
 	$(RM) $(GIT_TARNAME).tar.gz git-core_$(GIT_VERSION)-*.tar.gz
 	$(RM) $(htmldocs).tar.gz $(manpages).tar.gz
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 9eb22c4ef1..3bcc1ee57d 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -68,46 +68,6 @@ struct cmdname_help {
 	uint32_t category;
 };
 "
-if test -z "$2"
-then
-	define_categories "$1"
-	echo
-	print_command_list "$1"
-	exit 0
-fi
-
-echo "static const char *common_cmd_groups[] = {"
-
-grps=grps$$.tmp
-match=match$$.tmp
-trap "rm -f '$grps' '$match'" 0 1 2 3 15
-
-sed -n '
-	1,/^### common groups/b
-	/^### command list/q
-	/^#/b
-	/^[ 	]*$/b
-	h;s/^[^ 	][^ 	]*[ 	][ 	]*\(.*\)/	N_("\1"),/p
-	g;s/^\([^ 	][^ 	]*\)[ 	].*/\1/w '$grps'
-	' "$1"
-printf '};\n\n'
-
-n=0
-substnum=
-while read grp
-do
-	echo "^git-..*[ 	]$grp"
-	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
-	n=$(($n+1))
-done <"$grps" >"$match"
-
-printf 'static struct cmdname_help common_cmds[] = {\n'
-grep -f "$match" "$1" |
-sed 's/^git-//' |
-sort |
-while read cmd tags
-do
-	tag=$(echo "$tags" | sed "$substnum; s/[^0-9]//g")
-	echo "	{\"$cmd\", $(get_synopsis git-$cmd), $tag},"
-done
-echo "};"
+define_categories "$1"
+echo
+print_command_list "$1"
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 11/13] command-list.txt: documentation and guide line

This is intended to help anybody who needs to update command-list.txt. It gives a brief introduction of all attributes a command can take.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 command-list.txt | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)
Show changes to command-list.txt +44 −0
diff --git a/command-list.txt b/command-list.txt
index 99ddc231c1..9c70c69193 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -1,3 +1,47 @@
+# Command classification list
+# ---------------------------
+# All supported commands, builtin or external, must be described in
+# here. This info is used to list commands in various places. Each
+# command is on one line followed by one or more attributes.
+#
+# The first attribute group is mandatory and indicates the command
+# type. This group includes:
+#
+#   mainporcelain
+#   ancillarymanipulators
+#   ancillaryinterrogators
+#   foreignscminterface
+#   plumbingmanipulators
+#   plumbinginterrogators
+#   synchingrepositories
+#   synchelpers
+#   purehelpers
+#
+# The type names are self explanatory. But if you want to see what
+# command belongs to what group to get a better picture, have a look
+# at "git" man page, "GIT COMMANDS" section.
+#
+# Commands of type mainporcelain can also optionally have one of these
+# attributes:
+#
+#   init
+#   worktree
+#   info
+#   history
+#   remote
+#
+# These commands are considered "common" and will show up in "git
+# help" output in groups. Uncommon porcelain commands must not
+# specify any of these attributes.
+#
+# "complete" attribute is used to mark that the command should be
+# completable by git-completion.bash. Note that by default,
+# mainporcelain commands are completable so you don't need this
+# attribute.
+#
+# While not true commands, guides are also specified here, which can
+# only have "guide" attribute and nothing else.
+#
 ### command list (do not change this line, also do not change alignment)
 # command name                          category [category] [category]
 git-add                                 mainporcelain           worktree
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 10/13] help: use command-list.txt for the source of guides

The help command currently hard codes the list of guides and their summary in C. Let's move this list to command-list.txt. This lets us extract summary lines from Documentation/git*.txt. This also potentially lets us list guides in git.txt, but I'll leave that for now.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/gitattributes.txt        |  2 +-
 Documentation/gitmodules.txt           |  2 +-
 Documentation/gitrevisions.txt         |  2 +-
 Makefile                               |  2 +-
 builtin/help.c                         | 32 --------------------------
 command-list.txt                       | 16 +++++++++++++
 contrib/completion/git-completion.bash | 15 ++++++++----
 help.c                                 | 21 +++++++++++++----
 help.h                                 |  1 +
 t/t0012-help.sh                        |  6 +++++
 10 files changed, 54 insertions(+), 45 deletions(-)
Show changes to 10 files +54 −45

Documentation/gitattributes.txt, Documentation/gitmodules.txt, Documentation/gitrevisions.txt, Makefile, builtin/help.c, command-list.txt, contrib/completion/git-completion.bash, help.c, help.h, t/t0012-help.sh

diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt
index 1094fe2b5b..083c2f380d 100644
--- a/Documentation/gitattributes.txt
+++ b/Documentation/gitattributes.txt
@@ -3,7 +3,7 @@ gitattributes(5)
 
 NAME
 ----
-gitattributes - defining attributes per path
+gitattributes - Defining attributes per path
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitmodules.txt b/Documentation/gitmodules.txt
index db5d47eb19..4d63def206 100644
--- a/Documentation/gitmodules.txt
+++ b/Documentation/gitmodules.txt
@@ -3,7 +3,7 @@ gitmodules(5)
 
 NAME
 ----
-gitmodules - defining submodule properties
+gitmodules - Defining submodule properties
 
 SYNOPSIS
 --------
diff --git a/Documentation/gitrevisions.txt b/Documentation/gitrevisions.txt
index 27dec5b91d..1f6cceaefb 100644
--- a/Documentation/gitrevisions.txt
+++ b/Documentation/gitrevisions.txt
@@ -3,7 +3,7 @@ gitrevisions(7)
 
 NAME
 ----
-gitrevisions - specifying revisions and ranges for Git
+gitrevisions - Specifying revisions and ranges for Git
 
 SYNOPSIS
 --------
diff --git a/Makefile b/Makefile
index a60a78ee67..1efb751e46 100644
--- a/Makefile
+++ b/Makefile
@@ -1937,7 +1937,7 @@ $(BUILT_INS): git$X
 
 command-list.h: generate-cmdlist.sh command-list.txt
 
-command-list.h: $(wildcard Documentation/git-*.txt)
+command-list.h: $(wildcard Documentation/git*.txt)
 	$(QUIET_GEN)$(SHELL_PATH) ./generate-cmdlist.sh command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
diff --git a/builtin/help.c b/builtin/help.c
index 0e0af8426a..5727fb5e51 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -402,38 +402,6 @@ static void show_html_page(const char *git_cmd)
 	open_html(page_path.buf);
 }
 
-static struct {
-	const char *name;
-	const char *help;
-} common_guides[] = {
-	{ "attributes", N_("Defining attributes per path") },
-	{ "everyday", N_("Everyday Git With 20 Commands Or So") },
-	{ "glossary", N_("A Git glossary") },
-	{ "ignore", N_("Specifies intentionally untracked files to ignore") },
-	{ "modules", N_("Defining submodule properties") },
-	{ "revisions", N_("Specifying revisions and ranges for Git") },
-	{ "tutorial", N_("A tutorial introduction to Git (for version 1.5.1 or newer)") },
-	{ "workflows", N_("An overview of recommended workflows with Git") },
-};
-
-static void list_common_guides_help(void)
-{
-	int i, longest = 0;
-
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		if (longest < strlen(common_guides[i].name))
-			longest = strlen(common_guides[i].name);
-	}
-
-	puts(_("The common Git guides are:\n"));
-	for (i = 0; i < ARRAY_SIZE(common_guides); i++) {
-		printf("   %s   ", common_guides[i].name);
-		mput_char(' ', longest - strlen(common_guides[i].name));
-		puts(_(common_guides[i].help));
-	}
-	putchar('\n');
-}
-
 static const char *check_git_cmd(const char* cmd)
 {
 	char *alias;
diff --git a/command-list.txt b/command-list.txt
index 3bd23201a6..99ddc231c1 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -139,3 +139,19 @@ gitweb                                  ancillaryinterrogators
 git-whatchanged                         ancillaryinterrogators
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
+gitattributes                           guide
+gitcli                                  guide
+gitcore-tutorial                        guide
+gitcvs-migration                        guide
+gitdiffcore                             guide
+giteveryday                             guide
+gitglossary                             guide
+githooks                                guide
+gitignore                               guide
+gitmodules                              guide
+gitnamespaces                           guide
+gitrepository-layout                    guide
+gitrevisions                            guide
+gittutorial-2                           guide
+gittutorial                             guide
+gitworkflows                            guide
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 62ca8641f4..4e724a5b76 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1575,6 +1575,13 @@ _git_grep ()
 	__git_complete_refs
 }
 
+__git_all_guides=
+__git_compute_all_guides ()
+{
+	test -n "$__git_all_guides" ||
+	__git_all_guides=$(git --list-cmds=list-guide)
+}
+
 _git_help ()
 {
 	case "$cur" in
@@ -1584,11 +1591,9 @@ _git_help ()
 		;;
 	esac
 	__git_compute_all_commands
-	__gitcomp "$__git_all_commands $(__git_aliases)
-		attributes cli core-tutorial cvs-migration
-		diffcore everyday gitk glossary hooks ignore modules
-		namespaces repository-layout revisions tutorial tutorial-2
-		workflows
+	__git_compute_all_guides
+	__gitcomp "$__git_all_commands $(__git_aliases) $__git_all_guides
+		gitk
 		"
 }
 
diff --git a/help.c b/help.c
index c7df1d2338..23924dd300 100644
--- a/help.c
+++ b/help.c
@@ -39,12 +39,14 @@ static struct category_description main_categories[] = {
 	{ 0, NULL }
 };
 
-static const char *drop_prefix(const char *name)
+static const char *drop_prefix(const char *name, uint32_t category)
 {
 	const char *new_name;
 
 	if (skip_prefix(name, "git-", &new_name))
 		return new_name;
+	if (category == CAT_guide && skip_prefix(name, "git", &new_name))
+		return new_name;
 	return name;
 
 }
@@ -66,7 +68,7 @@ static void extract_cmds(struct cmdname_help **p_cmds, uint32_t mask)
 			continue;
 
 		cmds[nr] = *cmd;
-		cmds[nr].name = drop_prefix(cmd->name);
+		cmds[nr].name = drop_prefix(cmd->name, cmd->category);
 
 		nr++;
 	}
@@ -358,11 +360,22 @@ void list_cmds_by_category(struct string_list *list,
 	for (i = 0; i < n; i++) {
 		struct cmdname_help *cmd = command_list + i;
 
-		if (cmd->category & cat_id)
-			string_list_append(list, drop_prefix(cmd->name));
+		if (!(cmd->category & cat_id))
+			continue;
+		string_list_append(list, drop_prefix(cmd->name, cmd->category));
 	}
 }
 
+void list_common_guides_help(void)
+{
+	struct category_description catdesc[] = {
+		{ CAT_guide, N_("The common Git guides are:") },
+		{ 0, NULL }
+	};
+	print_cmd_by_category(catdesc);
+	putchar('\n');
+}
+
 void list_all_cmds_help(void)
 {
 	print_cmd_by_category(main_categories);
diff --git a/help.h b/help.h
index 40917fc38c..b2293e99be 100644
--- a/help.h
+++ b/help.h
@@ -20,6 +20,7 @@ static inline void mput_char(char c, unsigned int num)
 
 extern void list_common_cmds_help(void);
 extern void list_all_cmds_help(void);
+extern void list_common_guides_help(void);
 
 extern void list_all_main_cmds(struct string_list *list);
 extern void list_all_other_cmds(struct string_list *list);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 060df24c2d..bc27df7f38 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -66,6 +66,12 @@ test_expect_success 'git help' '
 	test_i18ngrep "^   commit " help.output &&
 	test_i18ngrep "^   fetch  " help.output
 '
+test_expect_success 'git help -g' '
+	git help -g >help.output &&
+	test_i18ngrep "^   attributes " help.output &&
+	test_i18ngrep "^   everyday   " help.output &&
+	test_i18ngrep "^   tutorial   " help.output
+'
 
 test_expect_success 'generate builtin list' '
 	git --list-cmds=builtins >builtins
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 12/13] completion: let git provide the completable command list

Instead of maintaining a separate list of command classification, which often could go out of date, let's centralize the information back in git.

While the function in git-completion.bash implies "list porcelain commands", that's not exactly what it does. It gets all commands (aka --list-cmds=main,others) then exclude certain non-porcelain ones. We could almost recreate this list two lists list-mainporcelain and others. The non-porcelain-but-included-anyway is added by the third category list-complete.

list-complete does not recreate exactly the command list before this patch though. The following commands are not part of neither list-mainporcelain nor list-complete and as a result no longer completes:

- annotate             obsolete, discouraged to use
- difftool-helper      not an end user command
- filter-branch        not often used
- get-tar-commit-id    not often used
- imap-send            not often used
- interpreter-trailers not for interactive use
- lost-found           obsolete
- p4                   too short and probably not often used (*)
- peek-remote          deprecated
- svn                  same category as p4 (*)
- tar-tree             obsolete
- verify-commit        not often used

Note that the current completion script incorrectly classifies filter-branch as porcelain and t9902 tests this behavior. We keep it this way in t9902 because this test does not really care which particular command is porcelain or plubmbing, they're just names.

(*) to be fair, send-email command which is in the same foreignscminterface group as svn and p4 does get completion, just because it's used by git and kernel development. So maybe should include them.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 command-list.txt                       |  37 ++++----
 contrib/completion/git-completion.bash | 117 ++++++-------------------
 t/t9902-completion.sh                  |   5 +-
 3 files changed, 48 insertions(+), 111 deletions(-)
Show changes to 3 files +48 −111

command-list.txt, contrib/completion/git-completion.bash, t/t9902-completion.sh

diff --git a/command-list.txt b/command-list.txt
index 9c70c69193..3e21ddfcfb 100644
--- a/command-list.txt
+++ b/command-list.txt
@@ -47,11 +47,11 @@
 git-add                                 mainporcelain           worktree
 git-am                                  mainporcelain
 git-annotate                            ancillaryinterrogators
-git-apply                               plumbingmanipulators
+git-apply                               plumbingmanipulators            complete
 git-archimport                          foreignscminterface
 git-archive                             mainporcelain
 git-bisect                              mainporcelain           info
-git-blame                               ancillaryinterrogators
+git-blame                               ancillaryinterrogators          complete
 git-branch                              mainporcelain           history
 git-bundle                              mainporcelain
 git-cat-file                            plumbinginterrogators
@@ -61,7 +61,7 @@ git-check-mailmap                       purehelpers
 git-checkout                            mainporcelain           history
 git-checkout-index                      plumbingmanipulators
 git-check-ref-format                    purehelpers
-git-cherry                              ancillaryinterrogators
+git-cherry                              ancillaryinterrogators          complete
 git-cherry-pick                         mainporcelain
 git-citool                              mainporcelain
 git-clean                               mainporcelain
@@ -69,7 +69,7 @@ git-clone                               mainporcelain           init
 git-column                              purehelpers
 git-commit                              mainporcelain           history
 git-commit-tree                         plumbingmanipulators
-git-config                              ancillarymanipulators
+git-config                              ancillarymanipulators           complete
 git-count-objects                       ancillaryinterrogators
 git-credential                          purehelpers
 git-credential-cache                    purehelpers
@@ -83,7 +83,7 @@ git-diff                                mainporcelain           history
 git-diff-files                          plumbinginterrogators
 git-diff-index                          plumbinginterrogators
 git-diff-tree                           plumbinginterrogators
-git-difftool                            ancillaryinterrogators
+git-difftool                            ancillaryinterrogators          complete
 git-fast-export                         ancillarymanipulators
 git-fast-import                         ancillarymanipulators
 git-fetch                               mainporcelain           remote
@@ -92,20 +92,20 @@ git-filter-branch                       ancillarymanipulators
 git-fmt-merge-msg                       purehelpers
 git-for-each-ref                        plumbinginterrogators
 git-format-patch                        mainporcelain
-git-fsck                                ancillaryinterrogators
+git-fsck                                ancillaryinterrogators          complete
 git-gc                                  mainporcelain
 git-get-tar-commit-id                   ancillaryinterrogators
 git-grep                                mainporcelain           info
 git-gui                                 mainporcelain
 git-hash-object                         plumbingmanipulators
-git-help                                ancillaryinterrogators
+git-help                                ancillaryinterrogators          complete
 git-http-backend                        synchingrepositories
 git-http-fetch                          synchelpers
 git-http-push                           synchelpers
 git-imap-send                           foreignscminterface
 git-index-pack                          plumbingmanipulators
 git-init                                mainporcelain           init
-git-instaweb                            ancillaryinterrogators
+git-instaweb                            ancillaryinterrogators          complete
 git-interpret-trailers                  purehelpers
 gitk                                    mainporcelain
 git-log                                 mainporcelain           info
@@ -119,12 +119,12 @@ git-merge-base                          plumbinginterrogators
 git-merge-file                          plumbingmanipulators
 git-merge-index                         plumbingmanipulators
 git-merge-one-file                      purehelpers
-git-mergetool                           ancillarymanipulators
+git-mergetool                           ancillarymanipulators           complete
 git-merge-tree                          ancillaryinterrogators
 git-mktag                               plumbingmanipulators
 git-mktree                              plumbingmanipulators
 git-mv                                  mainporcelain           worktree
-git-name-rev                            plumbinginterrogators
+git-name-rev                            plumbinginterrogators           complete
 git-notes                               mainporcelain
 git-p4                                  foreignscminterface
 git-pack-objects                        plumbingmanipulators
@@ -140,28 +140,29 @@ git-quiltimport                         foreignscminterface
 git-read-tree                           plumbingmanipulators
 git-rebase                              mainporcelain           history
 git-receive-pack                        synchelpers
-git-reflog                              ancillarymanipulators
-git-remote                              ancillarymanipulators
-git-repack                              ancillarymanipulators
-git-replace                             ancillarymanipulators
-git-request-pull                        foreignscminterface
+git-reflog                              ancillarymanipulators           complete
+git-remote                              ancillarymanipulators           complete
+git-repack                              ancillarymanipulators           complete
+git-replace                             ancillarymanipulators           complete
+git-request-pull                        foreignscminterface             complete
 git-rerere                              ancillaryinterrogators
 git-reset                               mainporcelain           worktree
 git-revert                              mainporcelain
 git-rev-list                            plumbinginterrogators
 git-rev-parse                           ancillaryinterrogators
 git-rm                                  mainporcelain           worktree
-git-send-email                          foreignscminterface
+git-send-email                          foreignscminterface             complete
 git-send-pack                           synchingrepositories
 git-shell                               synchelpers
 git-shortlog                            mainporcelain
 git-show                                mainporcelain           info
-git-show-branch                         ancillaryinterrogators
+git-show-branch                         ancillaryinterrogators          complete
 git-show-index                          plumbinginterrogators
 git-show-ref                            plumbinginterrogators
 git-sh-i18n                             purehelpers
 git-sh-setup                            purehelpers
 git-stash                               mainporcelain
+git-stage                                                               complete
 git-status                              mainporcelain           info
 git-stripspace                          purehelpers
 git-submodule                           mainporcelain
@@ -180,7 +181,7 @@ git-verify-commit                       ancillaryinterrogators
 git-verify-pack                         plumbinginterrogators
 git-verify-tag                          ancillaryinterrogators
 gitweb                                  ancillaryinterrogators
-git-whatchanged                         ancillaryinterrogators
+git-whatchanged                         ancillaryinterrogators          complete
 git-worktree                            mainporcelain
 git-write-tree                          plumbingmanipulators
 gitattributes                           guide
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 4e724a5b76..908692ea52 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -835,18 +835,30 @@ __git_complete_strategy ()
 }
 
 __git_commands () {
-	if test -n "${GIT_TESTING_COMMAND_COMPLETION:-}"
-	then
-		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
-	else
-		git --list-cmds=main,others
-	fi
+	case "$1" in
+	porcelain)
+		if test -n "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		then
+			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		else
+			git --list-cmds=list-mainporcelain,others,list-complete
+		fi
+		;;
+	all)
+		if test -n "$GIT_TESTING_ALL_COMMAND_LIST"
+		then
+			printf "%s" "$GIT_TESTING_ALL_COMMAND_LIST"
+		else
+			git --list-cmds=main,others
+		fi
+		;;
+	esac
 }
 
-__git_list_all_commands ()
+__git_list_commands ()
 {
 	local i IFS=" "$'\n'
-	for i in $(__git_commands)
+	for i in $(__git_commands $1)
 	do
 		case $i in
 		*--*)             : helper pattern;;
@@ -855,6 +867,11 @@ __git_list_all_commands ()
 	done
 }
 
+__git_list_all_commands ()
+{
+	__git_list_commands all
+}
+
 __git_all_commands=
 __git_compute_all_commands ()
 {
@@ -864,89 +881,7 @@ __git_compute_all_commands ()
 
 __git_list_porcelain_commands ()
 {
-	local i IFS=" "$'\n'
-	__git_compute_all_commands
-	for i in $__git_all_commands
-	do
-		case $i in
-		*--*)             : helper pattern;;
-		applymbox)        : ask gittus;;
-		applypatch)       : ask gittus;;
-		archimport)       : import;;
-		cat-file)         : plumbing;;
-		check-attr)       : plumbing;;
-		check-ignore)     : plumbing;;
-		check-mailmap)    : plumbing;;
-		check-ref-format) : plumbing;;
-		checkout-index)   : plumbing;;
-		column)           : internal helper;;
-		commit-tree)      : plumbing;;
-		count-objects)    : infrequent;;
-		credential)       : credentials;;
-		credential-*)     : credentials helper;;
-		cvsexportcommit)  : export;;
-		cvsimport)        : import;;
-		cvsserver)        : daemon;;
-		daemon)           : daemon;;
-		diff-files)       : plumbing;;
-		diff-index)       : plumbing;;
-		diff-tree)        : plumbing;;
-		fast-import)      : import;;
-		fast-export)      : export;;
-		fsck-objects)     : plumbing;;
-		fetch-pack)       : plumbing;;
-		fmt-merge-msg)    : plumbing;;
-		for-each-ref)     : plumbing;;
-		hash-object)      : plumbing;;
-		http-*)           : transport;;
-		index-pack)       : plumbing;;
-		init-db)          : deprecated;;
-		local-fetch)      : plumbing;;
-		ls-files)         : plumbing;;
-		ls-remote)        : plumbing;;
-		ls-tree)          : plumbing;;
-		mailinfo)         : plumbing;;
-		mailsplit)        : plumbing;;
-		merge-*)          : plumbing;;
-		mktree)           : plumbing;;
-		mktag)            : plumbing;;
-		pack-objects)     : plumbing;;
-		pack-redundant)   : plumbing;;
-		pack-refs)        : plumbing;;
-		parse-remote)     : plumbing;;
-		patch-id)         : plumbing;;
-		prune)            : plumbing;;
-		prune-packed)     : plumbing;;
-		quiltimport)      : import;;
-		read-tree)        : plumbing;;
-		receive-pack)     : plumbing;;
-		remote-*)         : transport;;
-		rerere)           : plumbing;;
-		rev-list)         : plumbing;;
-		rev-parse)        : plumbing;;
-		runstatus)        : plumbing;;
-		sh-setup)         : internal;;
-		shell)            : daemon;;
-		show-ref)         : plumbing;;
-		send-pack)        : plumbing;;
-		show-index)       : plumbing;;
-		ssh-*)            : transport;;
-		stripspace)       : plumbing;;
-		symbolic-ref)     : plumbing;;
-		unpack-file)      : plumbing;;
-		unpack-objects)   : plumbing;;
-		update-index)     : plumbing;;
-		update-ref)       : plumbing;;
-		update-server-info) : daemon;;
-		upload-archive)   : plumbing;;
-		upload-pack)      : plumbing;;
-		write-tree)       : plumbing;;
-		var)              : infrequent;;
-		verify-pack)      : infrequent;;
-		verify-tag)       : plumbing;;
-		*) echo $i;;
-		esac
-	done
+	__git_list_commands porcelain
 }
 
 __git_porcelain_commands=
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 1b34caa1e1..2f16679380 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -13,7 +13,7 @@ complete ()
 	return 0
 }
 
-# Be careful when updating this list:
+# Be careful when updating these lists:
 #
 # (1) The build tree may have build artifact from different branch, or
 #     the user's $PATH may have a random executable that may begin
@@ -30,7 +30,8 @@ complete ()
 #     completion for "git <TAB>", and a plumbing is excluded.  "add",
 #     "filter-branch" and "ls-files" are listed for this.
 
-GIT_TESTING_COMMAND_COMPLETION='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_ALL_COMMAND_LIST='add checkout check-attr filter-branch ls-files'
+GIT_TESTING_PORCELAIN_COMMAND_LIST='add checkout filter-branch'
 
 . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash"
 
-- 
2.17.0.705.g3525833791
SZEDER Gábor· May 11, 2018, 15:05 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v7 12/13] completion: let git provide the completable command list

On Thu, May 10, 2018 at 10:46 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

Show 28 quoted lines
> Instead of maintaining a separate list of command classification,
> which often could go out of date, let's centralize the information
> back in git.
>
> While the function in git-completion.bash implies "list porcelain
> commands", that's not exactly what it does. It gets all commands (aka
> --list-cmds=main,others) then exclude certain non-porcelain ones. We
> could almost recreate this list two lists list-mainporcelain and
> others. The non-porcelain-but-included-anyway is added by the third
> category list-complete.
>
> list-complete does not recreate exactly the command list before this
> patch though. The following commands are not part of neither
> list-mainporcelain nor list-complete and as a result no longer
> completes:
>
> - annotate             obsolete, discouraged to use
> - difftool-helper      not an end user command
> - filter-branch        not often used
> - get-tar-commit-id    not often used
> - imap-send            not often used
> - interpreter-trailers not for interactive use
> - lost-found           obsolete
> - p4                   too short and probably not often used (*)
> - peek-remote          deprecated
> - svn                  same category as p4 (*)
> - tar-tree             obsolete
> - verify-commit        not often used
'git name-rev' is plumbing as well.
I think this commit should be split into two:
  - first do the unequivocally beneficial thing and get rid of the
    long, hard-coded command list in __git_list_porcelain_commands(),
    while keeping its output unchanged,
  - then do the arguable thing and change the list of commands.
Show 16 quoted lines
> Note that the current completion script incorrectly classifies
> filter-branch as porcelain and t9902 tests this behavior. We keep it
> this way in t9902 because this test does not really care which
> particular command is porcelain or plubmbing, they're just names.
>
> (*) to be fair, send-email command which is in the same
> foreignscminterface group as svn and p4 does get completion, just
> because it's used by git and kernel development. So maybe should
> include them.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  command-list.txt                       |  37 ++++----
>  contrib/completion/git-completion.bash | 117 ++++++-------------------
>  t/t9902-completion.sh                  |   5 +-
>  3 files changed, 48 insertions(+), 111 deletions(-)
Show 36 quoted lines
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index 4e724a5b76..908692ea52 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -835,18 +835,30 @@ __git_complete_strategy ()
>  }
>
>  __git_commands () {
> -       if test -n "${GIT_TESTING_COMMAND_COMPLETION:-}"
> -       then
> -               printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
> -       else
> -               git --list-cmds=main,others
> -       fi
> +       case "$1" in
> +       porcelain)
> +               if test -n "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
> +               then
> +                       printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
> +               else
> +                       git --list-cmds=list-mainporcelain,others,list-complete
> +               fi
> +               ;;
> +       all)
> +               if test -n "$GIT_TESTING_ALL_COMMAND_LIST"
> +               then
> +                       printf "%s" "$GIT_TESTING_ALL_COMMAND_LIST"
> +               else
> +                       git --list-cmds=main,others
> +               fi
> +               ;;
> +       esac
>  }
>
> -__git_list_all_commands ()
> +__git_list_commands ()

Please add comments before these functions to document their mandatory argument.

Show 7 quoted lines
>  {
>         local i IFS=" "$'\n'
> -       for i in $(__git_commands)
> +       for i in $(__git_commands $1)
>         do
>                 case $i in
>                 *--*)             : helper pattern;;

Is this loop to exclude helper commands with doubledash in their name still necessary?

Duy Nguyen· May 13, 2018, 06:50 UTC · re: SZEDER Gábor · lore

Re: [PATCH v7 12/13] completion: let git provide the completable command list

On Fri, May 11, 2018 at 5:05 PM, SZEDER Gábor <szeder.dev@gmail.com> wrote:
Show 32 quoted lines
> On Thu, May 10, 2018 at 10:46 AM, Nguyễn Thái Ngọc Duy
> <pclouds@gmail.com> wrote:
>> Instead of maintaining a separate list of command classification,
>> which often could go out of date, let's centralize the information
>> back in git.
>>
>> While the function in git-completion.bash implies "list porcelain
>> commands", that's not exactly what it does. It gets all commands (aka
>> --list-cmds=main,others) then exclude certain non-porcelain ones. We
>> could almost recreate this list two lists list-mainporcelain and
>> others. The non-porcelain-but-included-anyway is added by the third
>> category list-complete.
>>
>> list-complete does not recreate exactly the command list before this
>> patch though. The following commands are not part of neither
>> list-mainporcelain nor list-complete and as a result no longer
>> completes:
>>
>> - annotate             obsolete, discouraged to use
>> - difftool-helper      not an end user command
>> - filter-branch        not often used
>> - get-tar-commit-id    not often used
>> - imap-send            not often used
>> - interpreter-trailers not for interactive use
>> - lost-found           obsolete
>> - p4                   too short and probably not often used (*)
>> - peek-remote          deprecated
>> - svn                  same category as p4 (*)
>> - tar-tree             obsolete
>> - verify-commit        not often used
>
> 'git name-rev' is plumbing as well.

So? name-rev remains completable like before and is not mentioned in the above list. Am I missing something?

Show 7 quoted lines
> I think this commit should be split into two:
>
>   - first do the unequivocally beneficial thing and get rid of the
>     long, hard-coded command list in __git_list_porcelain_commands(),
>     while keeping its output unchanged,
>
>   - then do the arguable thing and change the list of commands.

I will. Though the first commit still changes the output slightly because there are three commands not in command-list.txt. To keep the output unchanged, I would need to add them back in command-list.txt first (and find the right group for them) only to remove those lines later (two of them deprecated, the other does not even have a man page). It's not worth the effort.

Show 10 quoted lines
>>  {
>>         local i IFS=" "$'\n'
>> -       for i in $(__git_commands)
>> +       for i in $(__git_commands $1)
>>         do
>>                 case $i in
>>                 *--*)             : helper pattern;;
>
> Is this loop to exclude helper commands with doubledash in their name
> still necessary?

It is needed for __git_list_all_commands() because that one still essentially grabs "git help -a", which includes command--helpers.

-- 
Duy
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 13/13] completion: allow to customize the completable command list

By default we show porcelain, external commands and a couple others that are also popular. If you are not happy with this list, you can now customize it. See the big comment block for details.

PS. perhaps I should make aliases a group too, which makes it possible to _not_ complete aliases by omitting this special group in $GIT_COMPLETION_CMD_GROUPS

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/config.txt               | 10 ++++++++
 contrib/completion/git-completion.bash | 28 +++++++++++++++++++++-
 git.c                                  |  2 ++
 help.c                                 | 33 ++++++++++++++++++++++++++
 help.h                                 |  1 +
 5 files changed, 73 insertions(+), 1 deletion(-)
Show changes to 5 files +73 −1

Documentation/config.txt, contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/Documentation/config.txt b/Documentation/config.txt
index 2659153cb3..91f7eaed7b 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1343,6 +1343,16 @@ credential.<url>.*::
 credentialCache.ignoreSIGHUP::
 	Tell git-credential-cache--daemon to ignore SIGHUP, instead of quitting.
 
+completion.commands::
+	This is only used by git-completion.bash to add or remove
+	commands from the complete list. Normally only porcelain
+	commands and a few select others are in the complete list. You
+	can add more commands, separated by space, in this
+	variable. Prefixing the command with '-' will remove it from
+	the existing list.
++
+This variable should not be per-repository.
+
 include::diff-config.txt[]
 
 difftool.<tool>.path::
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 908692ea52..f7ca9a4d4f 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -38,6 +38,29 @@
 #
 #     When set to "1", do not include "DWIM" suggestions in git-checkout
 #     completion (e.g., completing "foo" when "origin/foo" exists).
+#
+#   GIT_COMPLETION_CMD_GROUPS
+#
+#     When set, "git --list-cmds=$GIT_COMPLETION_CMD_GROUPS" will be
+#     used to get the list of completable commands. The default is
+#     "mainporcelain,others,list-complete" (in English: all porcelain
+#     commands and external ones are included. Certain non-porcelain
+#     commands are also marked for completion in command-list.txt).
+#     You could for example complete all commands with
+#
+#         GIT_COMPLETION_CMD_GROUPS=main,others
+#
+#     Or you could go with main porcelain only and extra commands in
+#     the configuration variable completion.commands with
+#
+#         GIT_COMPLETION_CMD_GROUPS=mainporcelain,config
+#
+#     Or go completely custom group with
+#
+#         GIT_COMPLETION_CMD_GROUPS=config
+#
+#     Or you could even play with other command categories found in
+#     command-list.txt.
 
 case "$COMP_WORDBREAKS" in
 *:*) : great ;;
@@ -840,8 +863,11 @@ __git_commands () {
 		if test -n "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
 		then
 			printf "%s" "$GIT_TESTING_PORCELAIN_COMMAND_LIST"
+		elif test -n "$GIT_COMPLETION_CMD_GROUPS"
+		then
+			git --list-cmds="$GIT_COMPLETION_CMD_GROUPS"
 		else
-			git --list-cmds=list-mainporcelain,others,list-complete
+			git --list-cmds=list-mainporcelain,others,list-complete,config
 		fi
 		;;
 	all)
diff --git a/git.c b/git.c
index 4d5b8a9931..ea4feedd0b 100644
--- a/git.c
+++ b/git.c
@@ -60,6 +60,8 @@ static int list_cmds(const char *spec)
 			list_all_main_cmds(&list);
 		else if (match_token(spec, len, "others"))
 			list_all_other_cmds(&list);
+		else if (match_token(spec, len, "config"))
+			list_cmds_by_config(&list);
 		else if (len > 5 && !strncmp(spec, "list-", 5)) {
 			struct strbuf sb = STRBUF_INIT;
 
diff --git a/help.c b/help.c
index 23924dd300..abf87205b2 100644
--- a/help.c
+++ b/help.c
@@ -366,6 +366,39 @@ void list_cmds_by_category(struct string_list *list,
 	}
 }
 
+void list_cmds_by_config(struct string_list *list)
+{
+	const char *cmd_list;
+
+	/*
+	 * There's no actual repository setup at this point (and even
+	 * if there is, we don't really care; only global config
+	 * matters). If we accidentally set up a repository, it's ok
+	 * too since the caller (git --list-cmds=) should exit shortly
+	 * anyway.
+	 */
+	if (git_config_get_string_const("completion.commands", &cmd_list))
+		return;
+
+	string_list_sort(list);
+	string_list_remove_duplicates(list, 0);
+
+	while (*cmd_list) {
+		struct strbuf sb = STRBUF_INIT;
+		const char *p = strchrnul(cmd_list, ' ');
+
+		strbuf_add(&sb, cmd_list, p - cmd_list);
+		if (*cmd_list == '-')
+			string_list_remove(list, cmd_list + 1, 0);
+		else
+			string_list_insert(list, sb.buf);
+		strbuf_release(&sb);
+		while (*p == ' ')
+			p++;
+		cmd_list = p;
+	}
+}
+
 void list_common_guides_help(void)
 {
 	struct category_description catdesc[] = {
diff --git a/help.h b/help.h
index b2293e99be..3b38292a1b 100644
--- a/help.h
+++ b/help.h
@@ -26,6 +26,7 @@ extern void list_all_main_cmds(struct string_list *list);
 extern void list_all_other_cmds(struct string_list *list);
 extern void list_cmds_by_category(struct string_list *list,
 				  const char *category);
+extern void list_cmds_by_config(struct string_list *list);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 05/13] git.c: convert --list-* to --list-cmds=*

Even if these are hidden options, let's make them a bit more generic since we're introducing more listing types shortly. The code is structured to allow combining multiple listing types together because we will soon add more types the 'builtins'.

'parseopt' remains separate because it has separate (SPC) to match git-completion.bash needs and will not combine with others.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  | 37 +++++++++++++++++++++-----
 t/t0012-help.sh                        |  2 +-
 3 files changed, 33 insertions(+), 8 deletions(-)
Show changes to 3 files +33 −8

contrib/completion/git-completion.bash, git.c, t/t0012-help.sh

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a757073945..3556838759 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -3049,7 +3049,7 @@ __git_complete_common () {
 __git_cmds_with_parseopt_helper=
 __git_support_parseopt_helper () {
 	test -n "$__git_cmds_with_parseopt_helper" ||
-		__git_cmds_with_parseopt_helper="$(__git --list-parseopt-builtins)"
+		__git_cmds_with_parseopt_helper="$(__git --list-cmds=parseopt)"
 
 	case " $__git_cmds_with_parseopt_helper " in
 	*" $1 "*)
diff --git a/git.c b/git.c
index 3a89893712..cd85355d81 100644
--- a/git.c
+++ b/git.c
@@ -38,6 +38,30 @@ static int use_pager = -1;
 
 static void list_builtins(unsigned int exclude_option, char sep);
 
+static int match_token(const char *spec, int len, const char *token)
+{
+	int token_len = strlen(token);
+
+	return len == token_len && !strncmp(spec, token, token_len);
+}
+
+static int list_cmds(const char *spec)
+{
+	while (*spec) {
+		const char *sep = strchrnul(spec, ',');
+		int len = sep - spec;
+
+		if (match_token(spec, len, "builtins"))
+			list_builtins(0, '\n');
+		else
+			die(_("unsupported command listing type '%s'"), spec);
+		spec += len;
+		if (*spec == ',')
+			spec++;
+	}
+	return 0;
+}
+
 static void commit_pager_choice(void) {
 	switch (use_pager) {
 	case 0:
@@ -223,12 +247,13 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			}
 			(*argv)++;
 			(*argc)--;
-		} else if (!strcmp(cmd, "--list-builtins")) {
-			list_builtins(0, '\n');
-			exit(0);
-		} else if (!strcmp(cmd, "--list-parseopt-builtins")) {
-			list_builtins(NO_PARSEOPT, ' ');
-			exit(0);
+		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
+			if (!strcmp(cmd, "parseopt")) {
+				list_builtins(NO_PARSEOPT, ' ');
+				exit(0);
+			} else {
+				exit(list_cmds(cmd));
+			}
 		} else {
 			fprintf(stderr, _("unknown option: %s\n"), cmd);
 			usage(git_usage_string);
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index c096f33505..3c91a9024a 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -59,7 +59,7 @@ test_expect_success 'git help' '
 '
 
 test_expect_success 'generate builtin list' '
-	git --list-builtins >builtins
+	git --list-cmds=builtins >builtins
 '
 
 while read builtin
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 06/13] git --list-cmds: collect command list in a string_list

Instead of printing the command directly one by one, keep them in a list and print at the end. This allows more modification before we print out (e.g. sorting, removing duplicates or even excluding some items).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 git.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)
Show changes to git.c +17 −5
diff --git a/git.c b/git.c
index cd85355d81..376a59b97f 100644
--- a/git.c
+++ b/git.c
@@ -36,7 +36,7 @@ const char git_more_info_string[] =
 
 static int use_pager = -1;
 
-static void list_builtins(unsigned int exclude_option, char sep);
+static void list_builtins(struct string_list *list, unsigned int exclude_option);
 
 static int match_token(const char *spec, int len, const char *token)
 {
@@ -47,18 +47,24 @@ static int match_token(const char *spec, int len, const char *token)
 
 static int list_cmds(const char *spec)
 {
+	struct string_list list = STRING_LIST_INIT_DUP;
+	int i;
+
 	while (*spec) {
 		const char *sep = strchrnul(spec, ',');
 		int len = sep - spec;
 
 		if (match_token(spec, len, "builtins"))
-			list_builtins(0, '\n');
+			list_builtins(&list, 0);
 		else
 			die(_("unsupported command listing type '%s'"), spec);
 		spec += len;
 		if (*spec == ',')
 			spec++;
 	}
+	for (i = 0; i < list.nr; i++)
+		puts(list.items[i].string);
+	string_list_clear(&list, 0);
 	return 0;
 }
 
@@ -249,7 +255,13 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
 			(*argc)--;
 		} else if (skip_prefix(cmd, "--list-cmds=", &cmd)) {
 			if (!strcmp(cmd, "parseopt")) {
-				list_builtins(NO_PARSEOPT, ' ');
+				struct string_list list = STRING_LIST_INIT_DUP;
+				int i;
+
+				list_builtins(&list, NO_PARSEOPT);
+				for (i = 0; i < list.nr; i++)
+					printf("%s ", list.items[i].string);
+				string_list_clear(&list, 0);
 				exit(0);
 			} else {
 				exit(list_cmds(cmd));
@@ -533,14 +545,14 @@ int is_builtin(const char *s)
 	return !!get_builtin(s);
 }
 
-static void list_builtins(unsigned int exclude_option, char sep)
+static void list_builtins(struct string_list *out, unsigned int exclude_option)
 {
 	int i;
 	for (i = 0; i < ARRAY_SIZE(commands); i++) {
 		if (exclude_option &&
 		    (commands[i].option & exclude_option))
 			continue;
-		printf("%s%c", commands[i].cmd, sep);
+		string_list_append(out, commands[i].cmd);
 	}
 }
 
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 07/13] completion: implement and use --list-cmds=main,others

Instead of parsing "git help -a" output, which is tricky to get right, less elegant and also slow, make git provide the list in a machine-friendly form. This adds two separate listing types, main and others, instead of just "all" for more flexibility.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 contrib/completion/git-completion.bash |  2 +-
 git.c                                  |  4 ++++
 help.c                                 | 32 ++++++++++++++++++++++++++
 help.h                                 |  4 ++++
 4 files changed, 41 insertions(+), 1 deletion(-)
Show changes to 4 files +41 −1

contrib/completion/git-completion.bash, git.c, help.c, help.h

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 3556838759..62ca8641f4 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -839,7 +839,7 @@ __git_commands () {
 	then
 		printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
 	else
-		git help -a|egrep '^  [a-zA-Z0-9]'
+		git --list-cmds=main,others
 	fi
 }
 
diff --git a/git.c b/git.c
index 376a59b97f..10907f7266 100644
--- a/git.c
+++ b/git.c
@@ -56,6 +56,10 @@ static int list_cmds(const char *spec)
 
 		if (match_token(spec, len, "builtins"))
 			list_builtins(&list, 0);
+		else if (match_token(spec, len, "main"))
+			list_all_main_cmds(&list);
+		else if (match_token(spec, len, "others"))
+			list_all_other_cmds(&list);
 		else
 			die(_("unsupported command listing type '%s'"), spec);
 		spec += len;
diff --git a/help.c b/help.c
index 2d6a3157f8..d5ce9dfcbb 100644
--- a/help.c
+++ b/help.c
@@ -297,6 +297,38 @@ void list_common_cmds_help(void)
 	print_cmd_by_category(common_categories);
 }
 
+void list_all_main_cmds(struct string_list *list)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < main_cmds.cnt; i++)
+		string_list_append(list, main_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
+void list_all_other_cmds(struct string_list *list)
+{
+	struct cmdnames main_cmds, other_cmds;
+	int i;
+
+	memset(&main_cmds, 0, sizeof(main_cmds));
+	memset(&other_cmds, 0, sizeof(other_cmds));
+	load_command_list("git-", &main_cmds, &other_cmds);
+
+	for (i = 0; i < other_cmds.cnt; i++)
+		string_list_append(list, other_cmds.names[i]->name);
+
+	clean_cmdnames(&main_cmds);
+	clean_cmdnames(&other_cmds);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index b21d7c94e8..97e6c0965e 100644
--- a/help.h
+++ b/help.h
@@ -1,6 +1,8 @@
 #ifndef HELP_H
 #define HELP_H
 
+struct string_list;
+
 struct cmdnames {
 	int alloc;
 	int cnt;
@@ -17,6 +19,8 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_main_cmds(struct string_list *list);
+extern void list_all_other_cmds(struct string_list *list);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.705.g3525833791
SZEDER Gábor· May 11, 2018, 14:06 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH v7 07/13] completion: implement and use --list-cmds=main,others

On Thu, May 10, 2018 at 10:46 AM, Nguyễn Thái Ngọc Duy <pclouds@gmail.com> wrote:

> Instead of parsing "git help -a" output, which is tricky to get right,
> less elegant and also slow,

Is it tricky? It seems rather straightforward. Is it slow? Well, there is a pipe and an egrep, sure, but thanks to caching it's only run once, so basically doesn't matter.

OTOH you don't mention the most important reason, namely that the completion script contains a long hard-coded list of the names of all known plumbing commands to filter out, which is redundant with the categorization in 'command-list.txt', is a maintenance burden, and tends to get out-of-sync when new plumbing commands are added.

Show 108 quoted lines
> make git provide the list in a
> machine-friendly form. This adds two separate listing types, main and
> others, instead of just "all" for more flexibility.
>
> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
> ---
>  contrib/completion/git-completion.bash |  2 +-
>  git.c                                  |  4 ++++
>  help.c                                 | 32 ++++++++++++++++++++++++++
>  help.h                                 |  4 ++++
>  4 files changed, 41 insertions(+), 1 deletion(-)
>
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index 3556838759..62ca8641f4 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -839,7 +839,7 @@ __git_commands () {
>         then
>                 printf "%s" "${GIT_TESTING_COMMAND_COMPLETION}"
>         else
> -               git help -a|egrep '^  [a-zA-Z0-9]'
> +               git --list-cmds=main,others
>         fi
>  }
>
> diff --git a/git.c b/git.c
> index 376a59b97f..10907f7266 100644
> --- a/git.c
> +++ b/git.c
> @@ -56,6 +56,10 @@ static int list_cmds(const char *spec)
>
>                 if (match_token(spec, len, "builtins"))
>                         list_builtins(&list, 0);
> +               else if (match_token(spec, len, "main"))
> +                       list_all_main_cmds(&list);
> +               else if (match_token(spec, len, "others"))
> +                       list_all_other_cmds(&list);
>                 else
>                         die(_("unsupported command listing type '%s'"), spec);
>                 spec += len;
> diff --git a/help.c b/help.c
> index 2d6a3157f8..d5ce9dfcbb 100644
> --- a/help.c
> +++ b/help.c
> @@ -297,6 +297,38 @@ void list_common_cmds_help(void)
>         print_cmd_by_category(common_categories);
>  }
>
> +void list_all_main_cmds(struct string_list *list)
> +{
> +       struct cmdnames main_cmds, other_cmds;
> +       int i;
> +
> +       memset(&main_cmds, 0, sizeof(main_cmds));
> +       memset(&other_cmds, 0, sizeof(other_cmds));
> +       load_command_list("git-", &main_cmds, &other_cmds);
> +
> +       for (i = 0; i < main_cmds.cnt; i++)
> +               string_list_append(list, main_cmds.names[i]->name);
> +
> +       clean_cmdnames(&main_cmds);
> +       clean_cmdnames(&other_cmds);
> +}
> +
> +void list_all_other_cmds(struct string_list *list)
> +{
> +       struct cmdnames main_cmds, other_cmds;
> +       int i;
> +
> +       memset(&main_cmds, 0, sizeof(main_cmds));
> +       memset(&other_cmds, 0, sizeof(other_cmds));
> +       load_command_list("git-", &main_cmds, &other_cmds);
> +
> +       for (i = 0; i < other_cmds.cnt; i++)
> +               string_list_append(list, other_cmds.names[i]->name);
> +
> +       clean_cmdnames(&main_cmds);
> +       clean_cmdnames(&other_cmds);
> +}
> +
>  int is_in_cmdlist(struct cmdnames *c, const char *s)
>  {
>         int i;
> diff --git a/help.h b/help.h
> index b21d7c94e8..97e6c0965e 100644
> --- a/help.h
> +++ b/help.h
> @@ -1,6 +1,8 @@
>  #ifndef HELP_H
>  #define HELP_H
>
> +struct string_list;
> +
>  struct cmdnames {
>         int alloc;
>         int cnt;
> @@ -17,6 +19,8 @@ static inline void mput_char(char c, unsigned int num)
>  }
>
>  extern void list_common_cmds_help(void);
> +extern void list_all_main_cmds(struct string_list *list);
> +extern void list_all_other_cmds(struct string_list *list);
>  extern const char *help_unknown_cmd(const char *cmd);
>  extern void load_command_list(const char *prefix,
>                               struct cmdnames *main_cmds,
> --
> 2.17.0.705.g3525833791
>
SZEDER Gábor· May 11, 2018, 14:32 UTC · re: SZEDER Gábor · lore

Re: [PATCH v7 07/13] completion: implement and use --list-cmds=main,others

On Fri, May 11, 2018 at 4:06 PM, SZEDER Gábor <szeder.dev@gmail.com> wrote:
Show 5 quoted lines
> OTOH you don't mention the most important reason, namely that the
> completion script contains a long hard-coded list of the names of all
> known plumbing commands to filter out, which is redundant with the
> categorization in 'command-list.txt', is a maintenance burden, and
> tends to get out-of-sync when new plumbing commands are added.

Oops, sorry, got lost among the many patches, please just forget this paragraph.

Duy Nguyen· May 11, 2018, 16:33 UTC · re: SZEDER Gábor · lore

Re: [PATCH v7 07/13] completion: implement and use --list-cmds=main,others

On Fri, May 11, 2018 at 4:06 PM, SZEDER Gábor <szeder.dev@gmail.com> wrote:
Show 6 quoted lines
> On Thu, May 10, 2018 at 10:46 AM, Nguyễn Thái Ngọc Duy
> <pclouds@gmail.com> wrote:
>> Instead of parsing "git help -a" output, which is tricky to get right,
>> less elegant and also slow,
>
> Is it tricky?  It seems rather straightforward.

But there are traps. [1] is a reported one. And since you're parsing what's meant for human reader, this code could easily break in the future.

[1] https://public-inbox.org/git/CAPQz56bts8zFfUHyKJqnefQoH97L5TTA-k3be=5HsdeEbcMqOA@mail.gmail.com/
> Is it slow?  Well, there is a pipe and an egrep, sure, but thanks to
> caching it's only run once, so basically doesn't matter.

This I agree is an exaggeration. Ultimately though I am breaking down and providing the information pieces to __git_list_porcelain_commands() and it happens to benefit this one too. Perhaps I should rephrase my commit to say this.

-- 
Duy
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 08/13] git: support --list-cmds=list-<category>

This allows us to select any group of commands by a category defined in command-list.txt. This is an internal/hidden option so we don't have to be picky about the category name or worried about exposing too much.

This will be used later by git-completion.bash to retrieve certain command groups.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 generate-cmdlist.sh | 17 +++++++++++++++++
 git.c               |  7 +++++++
 help.c              | 23 +++++++++++++++++++++++
 help.h              |  2 ++
 4 files changed, 49 insertions(+)
Show changes to 4 files +49 −0

generate-cmdlist.sh, git.c, help.c, help.h

diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
index 3bcc1ee57d..8d6d8b45ce 100755
--- a/generate-cmdlist.sh
+++ b/generate-cmdlist.sh
@@ -45,6 +45,21 @@ define_categories () {
 	test "$bit" -gt 32 && die "Urgh.. too many categories?"
 }
 
+define_category_names () {
+	echo
+	echo "/* Category names */"
+	echo "static const char *category_names[] = {"
+	bit=0
+	category_list "$1" |
+	while read cat
+	do
+		echo "	\"$cat\", /* (1UL << $bit) */"
+		bit=$(($bit+1))
+	done
+	echo "	NULL"
+	echo "};"
+}
+
 print_command_list () {
 	echo "static struct cmdname_help command_list[] = {"
 
@@ -70,4 +85,6 @@ struct cmdname_help {
 "
 define_categories "$1"
 echo
+define_category_names "$1"
+echo
 print_command_list "$1"
diff --git a/git.c b/git.c
index 10907f7266..4d5b8a9931 100644
--- a/git.c
+++ b/git.c
@@ -60,6 +60,13 @@ static int list_cmds(const char *spec)
 			list_all_main_cmds(&list);
 		else if (match_token(spec, len, "others"))
 			list_all_other_cmds(&list);
+		else if (len > 5 && !strncmp(spec, "list-", 5)) {
+			struct strbuf sb = STRBUF_INIT;
+
+			strbuf_add(&sb, spec + 5, len - 5);
+			list_cmds_by_category(&list, sb.buf);
+			strbuf_release(&sb);
+		}
 		else
 			die(_("unsupported command listing type '%s'"), spec);
 		spec += len;
diff --git a/help.c b/help.c
index d5ce9dfcbb..1117f7d1d1 100644
--- a/help.c
+++ b/help.c
@@ -329,6 +329,29 @@ void list_all_other_cmds(struct string_list *list)
 	clean_cmdnames(&other_cmds);
 }
 
+void list_cmds_by_category(struct string_list *list,
+			   const char *cat)
+{
+	int i, n = ARRAY_SIZE(command_list);
+	uint32_t cat_id = 0;
+
+	for (i = 0; category_names[i]; i++) {
+		if (!strcmp(cat, category_names[i])) {
+			cat_id = 1UL << i;
+			break;
+		}
+	}
+	if (!cat_id)
+		die(_("unsupported command listing type '%s'"), cat);
+
+	for (i = 0; i < n; i++) {
+		struct cmdname_help *cmd = command_list + i;
+
+		if (cmd->category & cat_id)
+			string_list_append(list, drop_prefix(cmd->name));
+	}
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 97e6c0965e..734bba32d3 100644
--- a/help.h
+++ b/help.h
@@ -21,6 +21,8 @@ static inline void mput_char(char c, unsigned int num)
 extern void list_common_cmds_help(void);
 extern void list_all_main_cmds(struct string_list *list);
 extern void list_all_other_cmds(struct string_list *list);
+extern void list_cmds_by_category(struct string_list *list,
+				  const char *category);
 extern const char *help_unknown_cmd(const char *cmd);
 extern void load_command_list(const char *prefix,
 			      struct cmdnames *main_cmds,
-- 
2.17.0.705.g3525833791
Nguyễn Thái Ngọc Duy· May 10, 2018, 08:46 UTC · re: Nguyễn Thái Ngọc Duy · lore

[PATCH v7 09/13] help: add "-a --verbose" to list all commands with synopsis

This lists all recognized commands [1] by category. The group order follows closely git.txt.

[1] We may actually show commands that are not built (e.g. if you set NO_PERL you don't have git-instaweb but it's still listed here). I ignore the problem because on Linux a git package could be split anyway. The "git-core" package may not contain git-instaweb even if it's built because it may end up in a separate package. We can't know anyway.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 Documentation/git-help.txt |  4 +++-
 builtin/help.c             |  7 +++++++
 help.c                     | 16 ++++++++++++++++
 help.h                     |  2 ++
 t/t0012-help.sh            |  9 +++++++++
 5 files changed, 37 insertions(+), 1 deletion(-)
Show changes to 5 files +37 −1

Documentation/git-help.txt, builtin/help.c, help.c, help.h, t/t0012-help.sh

diff --git a/Documentation/git-help.txt b/Documentation/git-help.txt
index 40d328a4b3..a40fc38d8b 100644
--- a/Documentation/git-help.txt
+++ b/Documentation/git-help.txt
@@ -8,7 +8,7 @@ git-help - Display help information about Git
 SYNOPSIS
 --------
 [verse]
-'git help' [-a|--all] [-g|--guide]
+'git help' [-a|--all [--verbose]] [-g|--guide]
 	   [-i|--info|-m|--man|-w|--web] [COMMAND|GUIDE]
 
 DESCRIPTION
@@ -42,6 +42,8 @@ OPTIONS
 --all::
 	Prints all the available commands on the standard output. This
 	option overrides any given command or guide name.
+	When used with `--verbose` print description for all recognized
+	commands.
 
 -g::
 --guides::
diff --git a/builtin/help.c b/builtin/help.c
index 598867cfea..0e0af8426a 100644
--- a/builtin/help.c
+++ b/builtin/help.c
@@ -36,6 +36,7 @@ static const char *html_path;
 
 static int show_all = 0;
 static int show_guides = 0;
+static int verbose;
 static unsigned int colopts;
 static enum help_format help_format = HELP_FORMAT_NONE;
 static int exclude_guides;
@@ -48,6 +49,7 @@ static struct option builtin_help_options[] = {
 			HELP_FORMAT_WEB),
 	OPT_SET_INT('i', "info", &help_format, N_("show info page"),
 			HELP_FORMAT_INFO),
+	OPT__VERBOSE(&verbose, N_("print command description")),
 	OPT_END(),
 };
 
@@ -463,6 +465,11 @@ int cmd_help(int argc, const char **argv, const char *prefix)
 
 	if (show_all) {
 		git_config(git_help_config, NULL);
+		if (verbose) {
+			setup_pager();
+			list_all_cmds_help();
+			return 0;
+		}
 		printf(_("usage: %s%s"), _(git_usage_string), "\n\n");
 		load_command_list("git-", &main_cmds, &other_cmds);
 		list_commands(colopts, &main_cmds, &other_cmds);
diff --git a/help.c b/help.c
index 1117f7d1d1..c7df1d2338 100644
--- a/help.c
+++ b/help.c
@@ -27,6 +27,17 @@ static struct category_description common_categories[] = {
 	{ CAT_remote, N_("collaborate (see also: git help workflows)") },
 	{ 0, NULL }
 };
+static struct category_description main_categories[] = {
+	{ CAT_mainporcelain, N_("Main Porcelain Commands") },
+	{ CAT_ancillarymanipulators, N_("Ancillary Commands / Manipulators") },
+	{ CAT_ancillaryinterrogators, N_("Ancillary Commands / Interrogators") },
+	{ CAT_foreignscminterface, N_("Interacting with Others") },
+	{ CAT_plumbingmanipulators, N_("Low-level Commands / Manipulators") },
+	{ CAT_plumbinginterrogators, N_("Low-level Commands / Interrogators") },
+	{ CAT_synchingrepositories, N_("Low-level Commands / Synching Repositories") },
+	{ CAT_purehelpers, N_("Low-level Commands / Internal Helpers") },
+	{ 0, NULL }
+};
 
 static const char *drop_prefix(const char *name)
 {
@@ -352,6 +363,11 @@ void list_cmds_by_category(struct string_list *list,
 	}
 }
 
+void list_all_cmds_help(void)
+{
+	print_cmd_by_category(main_categories);
+}
+
 int is_in_cmdlist(struct cmdnames *c, const char *s)
 {
 	int i;
diff --git a/help.h b/help.h
index 734bba32d3..40917fc38c 100644
--- a/help.h
+++ b/help.h
@@ -19,6 +19,8 @@ static inline void mput_char(char c, unsigned int num)
 }
 
 extern void list_common_cmds_help(void);
+extern void list_all_cmds_help(void);
+
 extern void list_all_main_cmds(struct string_list *list);
 extern void list_all_other_cmds(struct string_list *list);
 extern void list_cmds_by_category(struct string_list *list,
diff --git a/t/t0012-help.sh b/t/t0012-help.sh
index 3c91a9024a..060df24c2d 100755
--- a/t/t0012-help.sh
+++ b/t/t0012-help.sh
@@ -25,6 +25,15 @@ test_expect_success "setup" '
 	EOF
 '
 
+# make sure to exercise these code paths, the output is a bit tricky
+# to verify
+test_expect_success 'basic help commands' '
+	git help >/dev/null &&
+	git help -a >/dev/null &&
+	git help -g >/dev/null &&
+	git help -av >/dev/null
+'
+
 test_expect_success "works for commands and guides by default" '
 	configure_help &&
 	git help status &&
-- 
2.17.0.705.g3525833791
Simon Ruderich· Apr 19, 2018, 10:37 UTC · re: Nguyễn Thái Ngọc Duy · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

Hello,

When running make -j$(nproc) with the current pu f9f8c1f765 (Merge branch 'hn/bisect-first-parent' into pu) I see the following error:

GIT_VERSION = 2.17.0.732.gf9f8c1f765
    * new build flags
    * new prefix flags
    GEN common-cmds.h
    * new link flags
    CC ident.o
    CC hex.o
    CC json-writer.o
./generate-cmdlist.sh: 73: ./generate-cmdlist.sh: Bad substitution
This doesn't occur on a non-parallel build.

Regards Simon

-- 
+ privacy is necessary
+ using gnupg http://gnupg.org
+ public key id: 0x92FEFDB7E44C32F9
SZEDER Gábor· Apr 19, 2018, 11:26 UTC · re: Simon Ruderich · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

On Thu, Apr 19, 2018 at 12:37 PM, Simon Ruderich <simon@ruderich.org> wrote:
Show 15 quoted lines
> When running make -j$(nproc) with the current pu f9f8c1f765
> (Merge branch 'hn/bisect-first-parent' into pu) I see the
> following error:
>
> GIT_VERSION = 2.17.0.732.gf9f8c1f765
>     * new build flags
>     * new prefix flags
>     GEN common-cmds.h
>     * new link flags
>     CC ident.o
>     CC hex.o
>     CC json-writer.o
> ./generate-cmdlist.sh: 73: ./generate-cmdlist.sh: Bad substitution
>
> This doesn't occur on a non-parallel build.
It does occur in non-parallel builds, too.
See:
  https://public-inbox.org/git/CAM0VKjkns+AsVyMSe2fxzT8a8oqYdNX3qO8mnw2juOgFC7LHYA@mail.gmail.com/
Simon Ruderich· Apr 20, 2018, 07:05 UTC · re: SZEDER Gábor · lore

Re: [PATCH/RFC 0/5] Keep all info in command-list.txt in git binary

On Thu, Apr 19, 2018 at 01:26:18PM +0200, SZEDER Gábor wrote:
Show 8 quoted lines
> On Thu, Apr 19, 2018 at 12:37 PM, Simon Ruderich wrote:
>> This doesn't occur on a non-parallel build.
>
> It does occur in non-parallel builds, too.
>
> See:
>
>   https://public-inbox.org/git/CAM0VKjkns+AsVyMSe2fxzT8a8oqYdNX3qO8mnw2juOgFC7LHYA@mail.gmail.com/

Thanks for the correction. I noticed it at the beginning of make -j run and incorrectly assummed it would occur quickly in the non-parallel run as well.

Regards Simon

-- 
+ privacy is necessary
+ using gnupg http://gnupg.org
+ public key id: 0x92FEFDB7E44C32F9

← back to recent threads