threads / patch / 40135

patchgenerate-cmdlist: re-implement as shell script

Subject: [PATCH] generate-cmdlist: re-implement as shell script

## tl;dr

5 messages between Aug 19, 2015 and Aug 20, 2015. Diffs are folded; open one to read it.

replies: 4people: 2as markdown or json

Eric Sunshine· Aug 19, 2015, 20:18 UTC · lore

527ec39 (generate-cmdlist: parse common group commands, 2015-05-21) replaced generate-cmdlist.sh with a more functional Perl version, generate-cmdlist.perl. The Perl version gleans named tags from a new "common groups" section in command-list.txt and recognizes those tags in "command list" section entries in place of the old 'common' tag. This allows git-help to, not only recognize, but also group common commands.

Although the tests require Perl, 527ec39 creates an unconditional dependence upon Perl in the build system itself, which can not be overridden with NO_PERL. Such a dependency may be undesirable; for instance, the 'git-lite' package in the FreeBSD ports tree is intended as a minimal Git installation (which may, for example, be useful on servers needing only local clone and update capability), which, historically, has not depended upon Perl[1].

Therefore, revive generate-cmdlist.sh and extend it to recognize "common groups" and its named tags. Retire generate-cmdlist.perl.

[1]: http://thread.gmane.org/gmane.comp.version-control.git/275905/focus=276132
Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---

In addition to reviving 527ec39^:generate-cmdlist.sh and extending it, I also re-indented it with tabs instead of spaces, so it's helpful to ignore whitespace changes when comparing the old and new versions of the shell script.

Makefile              |  4 ++--
 generate-cmdlist.perl | 50 --------------------------------------------------
 generate-cmdlist.sh   | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 52 insertions(+), 52 deletions(-)
 delete mode 100755 generate-cmdlist.perl
 create mode 100755 generate-cmdlist.sh
Show changes to 3 files +52 −52

Makefile, generate-cmdlist.perl, generate-cmdlist.sh

diff --git a/Makefile b/Makefile
index e39ca6c..ddfe7a1 100644
--- a/Makefile
+++ b/Makefile
@@ -1697,10 +1697,10 @@ $(BUILT_INS): git$X
 	ln -s $< $@ 2>/dev/null || \
 	cp $< $@
 
-common-cmds.h: generate-cmdlist.perl command-list.txt
+common-cmds.h: generate-cmdlist.sh command-list.txt
 
 common-cmds.h: $(wildcard Documentation/git-*.txt)
-	$(QUIET_GEN)$(PERL_PATH) generate-cmdlist.perl command-list.txt > $@+ && mv $@+ $@
+	$(QUIET_GEN)./generate-cmdlist.sh <command-list.txt >$@+ && mv $@+ $@
 
 SCRIPT_DEFINES = $(SHELL_PATH_SQ):$(DIFF_SQ):$(GIT_VERSION):\
 	$(localedir_SQ):$(NO_CURL):$(USE_GETTEXT_SCHEME):$(SANE_TOOL_PATH_SQ):\
diff --git a/generate-cmdlist.perl b/generate-cmdlist.perl
deleted file mode 100755
index 31516e3..0000000
--- a/generate-cmdlist.perl
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/usr/bin/perl
-use strict;
-use warnings;
-
-print <<"EOT";
-/* Automatically generated by $0 */
-
-struct cmdname_help {
-	char name[16];
-	char help[80];
-	unsigned char group;
-};
-
-static char *common_cmd_groups[] = {
-EOT
-
-my $n = 0;
-my %grp;
-while (<>) {
-	last if /^### command list/;
-	next if (1../^### common groups/) || /^#/ || /^\s*$/;
-	chop;
-	my ($k, $v) = split ' ', $_, 2;
-	$grp{$k} = $n++;
-	print "\tN_(\"$v\"),\n";
-}
-
-print "};\n\nstatic struct cmdname_help common_cmds[] = {\n";
-
-while (<>) {
-	next if /^#/ || /^\s*$/;
-	my @tags = split;
-	my $cmd = shift @tags;
-	for my $t (@tags) {
-		if (exists $grp{$t}) {
-			my $s;
-			open my $f, '<', "Documentation/$cmd.txt" or die;
-			while (<$f>) {
-				($s) = /^$cmd - (.+)$/;
-				last if $s;
-			}
-			close $f;
-			$cmd =~ s/^git-//;
-			print "\t{\"$cmd\", N_(\"$s\"), $grp{$t}},\n";
-			last;
-		}
-	}
-}
-
-print "};\n";
diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
new file mode 100755
index 0000000..1ac329d
--- /dev/null
+++ b/generate-cmdlist.sh
@@ -0,0 +1,50 @@
+#!/bin/sh
+
+echo "/* Automatically generated by $0 */
+struct cmdname_help {
+	char name[16];
+	char help[80];
+	unsigned char group;
+};
+
+static const char *common_cmd_groups[] = {"
+
+tmp=cmdgrps$$.tmp
+trap "rm -fr $tmp" 0 1 2 3 15
+
+sed -n '
+	1,/^### common groups/b
+	/^### command list/q
+	/^#/d; /^[ 	]*$/b
+	h;s/^[^ 	][^ 	]*[ 	][ 	]*\(.*\)/	N_("\1"),/p
+	g;s/^\([^ 	][^ 	]*\)[ 	].*/\1/w '$tmp'
+	'
+printf '};\n\n'
+
+n=0
+matchgrp=
+substnum=
+while read grp
+do
+	matchgrp="$matchgrp${matchgrp:+
+}^git-..*[ 	]$grp"
+	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
+	n=$(($n+1))
+done <$tmp
+
+printf 'static struct cmdname_help common_cmds[] = {\n'
+grep "$matchgrp" |
+sed 's/^git-//' |
+sort |
+while read cmd tags
+do
+	tag=$(echo $tags | sed "$substnum; s/[^0-9]//g")
+	sed -n '
+		/^NAME/,/git-'"$cmd"'/H
+		${
+			x
+			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
+			p
+		}' "Documentation/git-$cmd.txt"
+done
+echo "};"
-- 
2.5.0.457.gab17608
Junio C Hamano· Aug 19, 2015, 22:38 UTC · re: Eric Sunshine · lore

Re: [PATCH] generate-cmdlist: re-implement as shell script

Eric Sunshine <sunshine@sunshineco.com> writes:
> In addition to reviving 527ec39^:generate-cmdlist.sh and extending it, I
> also re-indented it with tabs instead of spaces, so it's helpful to
> ignore whitespace changes when comparing the old and new versions of the
> shell script.

Hmph. Perhaps we can view it as part of reverting 527ec39 and then redoing it in shell. The way the shell script accumulates matchgrp variable (i.e. the literal LF in ${var:+string}) makes me feel some possible portability scare, which might be solved in a more stupid (i.e. not giving various reimplementations of Bourne shells a chance to screw it up) way by using another temporary file, but other than that the resurrected script looks OK to me.

Thanks.
Show 56 quoted lines
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> new file mode 100755
> index 0000000..1ac329d
> --- /dev/null
> +++ b/generate-cmdlist.sh
> @@ -0,0 +1,50 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by $0 */
> +struct cmdname_help {
> +	char name[16];
> +	char help[80];
> +	unsigned char group;
> +};
> +
> +static const char *common_cmd_groups[] = {"
> +
> +tmp=cmdgrps$$.tmp
> +trap "rm -fr $tmp" 0 1 2 3 15
> +
> +sed -n '
> +	1,/^### common groups/b
> +	/^### command list/q
> +	/^#/d; /^[ 	]*$/b
> +	h;s/^[^ 	][^ 	]*[ 	][ 	]*\(.*\)/	N_("\1"),/p
> +	g;s/^\([^ 	][^ 	]*\)[ 	].*/\1/w '$tmp'
> +	'
> +printf '};\n\n'
> +
> +n=0
> +matchgrp=
> +substnum=
> +while read grp
> +do
> +	matchgrp="$matchgrp${matchgrp:+
> +}^git-..*[ 	]$grp"
> +	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
> +	n=$(($n+1))
> +done <$tmp
> +
> +printf 'static struct cmdname_help common_cmds[] = {\n'
> +grep "$matchgrp" |
> +sed 's/^git-//' |
> +sort |
> +while read cmd tags
> +do
> +	tag=$(echo $tags | sed "$substnum; s/[^0-9]//g")
> +	sed -n '
> +		/^NAME/,/git-'"$cmd"'/H
> +		${
> +			x
> +			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
> +			p
> +		}' "Documentation/git-$cmd.txt"
> +done
> +echo "};"
Eric Sunshine· Aug 20, 2015, 18:11 UTC · re: Junio C Hamano · lore

Re: [PATCH] generate-cmdlist: re-implement as shell script

On Wed, Aug 19, 2015 at 6:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 8 quoted lines
> Eric Sunshine <sunshine@sunshineco.com> writes:
>> In addition to reviving 527ec39^:generate-cmdlist.sh and extending it, I
>> also re-indented it with tabs instead of spaces, so it's helpful to
>> ignore whitespace changes when comparing the old and new versions of the
>> shell script.
>
> Hmph.  Perhaps we can view it as part of reverting 527ec39 and then
> redoing it in shell.

I'm having trouble understanding. Are you asking that this be presented as a series of patches which first revert 527ec39, then do the whitespace cleanup, and then augment the script for the extended functionality? If so, it's a bit problematic because the original script still expects the 'common' tag to be present, but that tag was removed by the subsequent patch 2f5b495 (command-list.txt: drop the "common" tag, 2015-05-21). So, reverting 527ec39 would also require reverting 2f5b495.

Show 5 quoted lines
> The way the shell script accumulates matchgrp
> variable (i.e. the literal LF in ${var:+string}) makes me feel some
> possible portability scare, which might be solved in a more stupid
> (i.e. not giving various reimplementations of Bourne shells a chance
> to screw it up) way by using another temporary file[...]

In addition to the literal newline and the temporary file, other options I considered included assigning the newline to a variable and then interpolating that variable (${var:+$LF}), or expanding the list of patterns into a set of '-e' arguments for grep. But, I think I'll just go with the temporary file.

Junio C Hamano· Aug 20, 2015, 17:24 UTC · re: Eric Sunshine · lore

Re: [PATCH] generate-cmdlist: re-implement as shell script

Eric Sunshine <sunshine@sunshineco.com> writes:
Show 28 quoted lines
> diff --git a/generate-cmdlist.sh b/generate-cmdlist.sh
> new file mode 100755
> index 0000000..1ac329d
> --- /dev/null
> +++ b/generate-cmdlist.sh
> @@ -0,0 +1,50 @@
> +#!/bin/sh
> +
> +echo "/* Automatically generated by $0 */
> +struct cmdname_help {
> +	char name[16];
> +	char help[80];
> +	unsigned char group;
> +};
> +
> +static const char *common_cmd_groups[] = {"
> +
> +tmp=cmdgrps$$.tmp
> +trap "rm -fr $tmp" 0 1 2 3 15
> +
> +sed -n '
> +	1,/^### common groups/b
> +	/^### command list/q
> +	/^#/d; /^[ 	]*$/b
> +	h;s/^[^ 	][^ 	]*[ 	][ 	]*\(.*\)/	N_("\1"),/p
> +	g;s/^\([^ 	][^ 	]*\)[ 	].*/\1/w '$tmp'
> +	'
> +printf '};\n\n'

Unfortunately, this does not seem to work for me. Even though sed stops reading after seeing the "### command list" line, I suspect that its stdin buffer has been filled with other lines that follow it from the input to the buffer size, consuming what you meant to feed the later 'grep $matchgrp"' with.

This is a one-time thing, so I do not mind to update the Makefile so that it does not feed command-list.txt from the standard input but gives the path as "$1" to this script.

Show 27 quoted lines
> +n=0
> +matchgrp=
> +substnum=
> +while read grp
> +do
> +	matchgrp="$matchgrp${matchgrp:+
> +}^git-..*[ 	]$grp"
> +	substnum="$substnum${substnum:+;}s/[ 	]$grp/$n/"
> +	n=$(($n+1))
> +done <$tmp
> +
> +printf 'static struct cmdname_help common_cmds[] = {\n'
> +grep "$matchgrp" |
> +sed 's/^git-//' |
> +sort |
> +while read cmd tags
> +do
> +	tag=$(echo $tags | sed "$substnum; s/[^0-9]//g")
> +	sed -n '
> +		/^NAME/,/git-'"$cmd"'/H
> +		${
> +			x
> +			s/.*git-'"$cmd"' - \(.*\)/	{"'"$cmd"'", N_("\1"), '$tag'},/
> +			p
> +		}' "Documentation/git-$cmd.txt"
> +done
> +echo "};"
Eric Sunshine· Aug 20, 2015, 18:18 UTC · re: Junio C Hamano · lore

Re: [PATCH] generate-cmdlist: re-implement as shell script

On Thu, Aug 20, 2015 at 1:24 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 19 quoted lines
> Eric Sunshine <sunshine@sunshineco.com> writes:
>> +sed -n '
>> +     1,/^### common groups/b
>> +     /^### command list/q
>> +     /^#/d; /^[      ]*$/b
>> +     h;s/^[^         ][^     ]*[     ][      ]*\(.*\)/       N_("\1"),/p
>> +     g;s/^\([^       ][^     ]*\)[   ].*/\1/w '$tmp'
>> +     '
>> +printf '};\n\n'
>
> Unfortunately, this does not seem to work for me.  Even though sed
> stops reading after seeing the "### command list" line, I suspect
> that its stdin buffer has been filled with other lines that follow
> it from the input to the buffer size, consuming what you meant to
> feed the later 'grep $matchgrp"' with.
>
> This is a one-time thing, so I do not mind to update the Makefile
> so that it does not feed command-list.txt from the standard input
> but gives the path as "$1" to this script.

The original generate-cmdlist.sh doesn't take any arguments and just hardcodes "command-list.txt". Feeding it instead on stdin seemed a nice way to avoid reading the file twice, but alas is too fragile. I don't mind passing it as an argument either, or just hardcoding it again (though my preference leans toward passing it as an argument).

← back to recent threads