git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [RFC/PATCH 1/5] Makefile: move long inline shell loops in "install" into helper

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Nov 12, 2018, 14:42 UTC
Message-ID
<87h8gm2w8n.fsf@evledraar.gmail.com>
In-Reply-To
<nycvar.QRO.7.76.6.1811121449080.39@tvgsbejvaqbjf.bet>
On Mon, Nov 12 2018, Johannes Schindelin wrote:
Show 236 quoted lines
> Hi,
>
> On Fri, 2 Nov 2018, Ævar Arnfjörð Bjarmason wrote:
>
>> Move a 37 line for-loop mess out of "install" and into a helper
>> script. This started out fairly innocent but over the years has grown
>> into a hard-to-maintain monster, and my recent ad874608d8 ("Makefile:
>> optionally symlink libexec/git-core binaries to bin/git", 2018-03-13)
>> certainly didn't help.
>>
>> The shell code is ported pretty much as-is (with getopts added), it'll
>> be fixed & prettified in subsequent commits.
>>
>> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>> ---
>>  Makefile         | 52 ++++++++--------------------
>>  install_programs | 89 ++++++++++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 103 insertions(+), 38 deletions(-)
>>  create mode 100755 install_programs
>>
>> diff --git a/Makefile b/Makefile
>> index bbfbb4292d..aa6ca1fa68 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -2808,44 +2808,20 @@ endif
>>  	bindir=$$(cd '$(DESTDIR_SQ)$(bindir_SQ)' && pwd) && \
>>  	execdir=$$(cd '$(DESTDIR_SQ)$(gitexec_instdir_SQ)' && pwd) && \
>>  	destdir_from_execdir_SQ=$$(echo '$(gitexecdir_relative_SQ)' | sed -e 's|[^/][^/]*|..|g') && \
>> -	{ test "$$bindir/" = "$$execdir/" || \
>> -	  for p in git$X $(filter $(install_bindir_programs),$(ALL_PROGRAMS)); do \
>> -		$(RM) "$$execdir/$$p" && \
>> -		test -n "$(INSTALL_SYMLINKS)" && \
>> -		ln -s "$$destdir_from_execdir_SQ/$(bindir_relative_SQ)/$$p" "$$execdir/$$p" || \
>> -		{ test -z "$(NO_INSTALL_HARDLINKS)$(NO_CROSS_DIRECTORY_HARDLINKS)" && \
>> -		  ln "$$bindir/$$p" "$$execdir/$$p" 2>/dev/null || \
>> -		  cp "$$bindir/$$p" "$$execdir/$$p" || exit; } \
>> -	  done; \
>> -	} && \
>> -	for p in $(filter $(install_bindir_programs),$(BUILT_INS)); do \
>> -		$(RM) "$$bindir/$$p" && \
>> -		test -n "$(INSTALL_SYMLINKS)" && \
>> -		ln -s "git$X" "$$bindir/$$p" || \
>> -		{ test -z "$(NO_INSTALL_HARDLINKS)" && \
>> -		  ln "$$bindir/git$X" "$$bindir/$$p" 2>/dev/null || \
>> -		  ln -s "git$X" "$$bindir/$$p" 2>/dev/null || \
>> -		  cp "$$bindir/git$X" "$$bindir/$$p" || exit; } \
>> -	done && \
>> -	for p in $(BUILT_INS); do \
>> -		$(RM) "$$execdir/$$p" && \
>> -		test -n "$(INSTALL_SYMLINKS)" && \
>> -		ln -s "$$destdir_from_execdir_SQ/$(bindir_relative_SQ)/git$X" "$$execdir/$$p" || \
>> -		{ test -z "$(NO_INSTALL_HARDLINKS)" && \
>> -		  ln "$$execdir/git$X" "$$execdir/$$p" 2>/dev/null || \
>> -		  ln -s "git$X" "$$execdir/$$p" 2>/dev/null || \
>> -		  cp "$$execdir/git$X" "$$execdir/$$p" || exit; } \
>> -	done && \
>> -	remote_curl_aliases="$(REMOTE_CURL_ALIASES)" && \
>> -	for p in $$remote_curl_aliases; do \
>> -		$(RM) "$$execdir/$$p" && \
>> -		test -n "$(INSTALL_SYMLINKS)" && \
>> -		ln -s "git-remote-http$X" "$$execdir/$$p" || \
>> -		{ test -z "$(NO_INSTALL_HARDLINKS)" && \
>> -		  ln "$$execdir/git-remote-http$X" "$$execdir/$$p" 2>/dev/null || \
>> -		  ln -s "git-remote-http$X" "$$execdir/$$p" 2>/dev/null || \
>> -		  cp "$$execdir/git-remote-http$X" "$$execdir/$$p" || exit; } \
>> -	done && \
>
> This indeed looks like a mess...
>
>> +	./install_programs \
>> +		--X="$$X" \
>> +		--RM="$(RM)" \
>> +		--bindir="$$bindir" \
>> +		--bindir-relative="$(bindir_relative_SQ)" \
>> +		--execdir="$$execdir" \
>> +		--destdir-from-execdir="$$destdir_from_execdir_SQ" \
>> +		--flag-install-symlinks="$(INSTALL_SYMLINKS)" \
>> +		--flag-no-install-hardlinks="$(NO_INSTALL_HARDLINKS)" \
>> +		--flag-no-cross-directory-hardlinks="$(NO_CROSS_DIRECTORY_HARDLINKS)" \
>> +		--list-bindir-standalone="git$X $(filter $(install_bindir_programs),$(ALL_PROGRAMS))" \
>> +		--list-bindir-git-dashed="$(filter $(install_bindir_programs),$(BUILT_INS))" \
>> +		--list-execdir-git-dashed="$(BUILT_INS)" \
>> +		--list-execdir-curl-aliases="$(REMOTE_CURL_ALIASES)" && \
>>  	./check_bindir "z$$bindir" "z$$execdir" "$$bindir/git-add$X"
>>
>>  .PHONY: install-gitweb install-doc install-man install-man-perl install-html install-info install-pdf
>> diff --git a/install_programs b/install_programs
>> new file mode 100755
>> index 0000000000..e287108112
>> --- /dev/null
>> +++ b/install_programs
>> @@ -0,0 +1,89 @@
>> +#!/bin/sh
>> +
>> +while test $# != 0
>> +do
>> +	case "$1" in
>> +	--X=*)
>> +		X="${1#--X=}"
>> +		;;
>> +	--RM=*)
>> +		RM="${1#--RM=}"
>> +		;;
>> +	--bindir=*)
>> +		bindir="${1#--bindir=}"
>> +		;;
>> +	--bindir-relative=*)
>> +		bindir_relative="${1#--bindir-relative=}"
>> +		;;
>> +	--execdir=*)
>> +		execdir="${1#--execdir=}"
>> +		;;
>> +	--destdir-from-execdir=*)
>> +		destdir_from_execdir="${1#--destdir-from-execdir=}"
>> +		;;
>> +	--flag-install-symlinks=*)
>> +		INSTALL_SYMLINKS="${1#--flag-install-symlinks=}"
>> +		;;
>> +	--flag-no-install-hardlinks=*)
>> +		NO_INSTALL_HARDLINKS="${1#--flag-no-install-hardlinks=}"
>> +		;;
>> +	--flag-no-cross-directory-hardlinks=*)
>> +		NO_CROSS_DIRECTORY_HARDLINKS="${1#--flag-no-cross-directory-hardlinks=}"
>> +		;;
>> +	--list-bindir-standalone=*)
>> +		list_bindir_standalone="${1#--list-bindir-standalone=}"
>> +		;;
>> +	--list-bindir-git-dashed=*)
>> +		list_bindir_git_dashed="${1#--list-bindir-git-dashed=}"
>> +		;;
>> +	--list-execdir-git-dashed=*)
>> +		list_execdir_git_dashed="${1#--list-execdir-git-dashed=}"
>> +		;;
>> +	--list-execdir-curl-aliases=*)
>> +		list_execdir_curl_aliases="${1#--list-execdir-curl-aliases=}"
>> +		;;
>> +
>> +	*)
>> +		echo "Unknown option $1"
>> +		exit 1
>> +		;;
>> +	esac
>> +	shift
>> +done &&
>> +{ test "$bindir/" = "$execdir/" ||
>> +  for p in $list_bindir_standalone; do
>> +	$RM "$execdir/$p" &&
>> +	test -n "$INSTALL_SYMLINKS" &&
>> +	ln -s "$destdir_from_execdir/$bindir_relative/$p" "$execdir/$p" ||
>> +	{ test -z "$NO_INSTALL_HARDLINKS$NO_CROSS_DIRECTORY_HARDLINKS" &&
>> +	  ln "$bindir/$p" "$execdir/$p" 2>/dev/null ||
>> +	  cp "$bindir/$p" "$execdir/$p" || exit; }
>> +  done;
>> +} &&
>> +for p in $list_bindir_git_dashed; do
>> +	$RM "$bindir/$p" &&
>> +	test -n "$INSTALL_SYMLINKS" &&
>> +	ln -s "git$X" "$bindir/$p" ||
>> +	{ test -z "$NO_INSTALL_HARDLINKS" &&
>> +	  ln "$bindir/git$X" "$bindir/$p" 2>/dev/null ||
>> +	  ln -s "git$X" "$bindir/$p" 2>/dev/null ||
>> +	  cp "$bindir/git$X" "$bindir/$p" || exit; }
>> +done &&
>> +for p in $list_execdir_git_dashed; do
>> +	$RM "$execdir/$p" &&
>> +	test -n "$INSTALL_SYMLINKS" &&
>> +	ln -s "$destdir_from_execdir/$bindir_relative/git$X" "$execdir/$p" ||
>> +	{ test -z "$NO_INSTALL_HARDLINKS" &&
>> +	  ln "$execdir/git$X" "$execdir/$p" 2>/dev/null ||
>> +	  ln -s "git$X" "$execdir/$p" 2>/dev/null ||
>> +	  cp "$execdir/git$X" "$execdir/$p" || exit; }
>> +done &&
>> +for p in $list_execdir_curl_aliases; do
>> +	$RM "$execdir/$p" &&
>> +	test -n "$INSTALL_SYMLINKS" &&
>> +	ln -s "git-remote-http$X" "$execdir/$p" ||
>> +	{ test -z "$NO_INSTALL_HARDLINKS" &&
>> +	  ln "$execdir/git-remote-http$X" "$execdir/$p" 2>/dev/null ||
>> +	  ln -s "git-remote-http$X" "$execdir/$p" 2>/dev/null ||
>> +	  cp "$execdir/git-remote-http$X" "$execdir/$p" || exit; }
>> +done
>
> ... but so does this. I would be very surprised if these four very
> similar-looking constructs could not be refactored into a single shell
> script that is then called four times with different parameters.
>
> Something like
>
> 	#!/bin/sh
>
> 	from=
> 	while case "$1" in
> 	--no-hardlinks)
> 		NO_INSTALL_HARDLINKS=t
> 		;;
> 	--from=*)
> 		from="${1#*=}"
> 		;;
> 	*)
> 		break
> 		;;
> 	esac; do
> 		shift
> 	done
>
> 	test $# -gt 3 || {
> 		echo "Usage: $0 [--no-hardlinks] <from-dir> <to-dir> <file>..." >&2
> 		exit 1
> 	}
>
> 	fromdir="$1"
> 	todir="$2"
> 	shift
> 	shift
>
> 	for p in "$@"
> 	do
> 		$RM "$todir/$p" &&
> 		test -n "$INSTALL_SYMLINKS" &&
> 		ln -s "$fromdir/${from:-$p}" "$todir/$p" ||
> 		{ test -z "$NO_INSTALL_HARDLINKS" &&
> 		  ln "$fromdir/${from:-$p}" "$todir/$p" ||
> 		  ln -s "$fromdir/${from:-$p}" "$todir/$p" ||
> 		  cp "$fromdir/${from:-$p}" "$todir/$p" || exit; }
> 	done
>
> and then calling it using
>
> 	test "$bindir/" = "$execdir/" ||
> 	link-or-copy ${NO_CROSS_DIRECTORY_HARDLINKS:+--no-hardlinks} \
> 		"$bindir" "$execdir" $list_bindir_standalone
> 	link-or-copy --from=git$X "$bindir" "$bindir" $list_bindir_git_dashed
> 	link-or-copy --from=git$X "$bindir" "$execdir" $list_bindir_git_dashed
> 	link-or-copy --from=git-remote-http$X "$bindir" "$execdir" $list_execdir_curl_aliases
>
> That would at least DRY up this mess a bit.

I'll try for a non-RFC re-submission of this which won't have the 5/5 NO_INSTALL_BUILTIN_EXECDIR_ALIASES (for now) which'll hopefully be ready for inclusion.

I tried to fold up some of these special cases into one thing before, but managing the exceptions gets messier to read in my opinion than just having some duplication.

But more to the point, between your suggestion and Junio's to do all of this with some make construct: Yeah we should make it awesome, but I think a logical first step for a patch series like this is to just as-is use the existing logic we have now with some minor fixes for bugs.

We can refactor this later, I'm mainly interested in moving this over to a *.sh first so that process is less of a mess, fixing some minor bugs, and not getting a first iteration of improvements stuck on a much bigger review for "I rewrote the way the whole install process works" series.

Previous: Johannes SchindelinNext: Johannes Schindelin
Message 34 of 53 in “What's cooking in git.git (Mar 2018, #02; Tue, 6)”
  1. Junio C HamanoMar 6, 2018
  2. Johannes SchindelinMar 7, 2018
  3. Ævar Arnfjörð BjarmasonMar 8, 2018
  4. Daniel JacquesMar 8, 2018
  5. Why don't we symlink libexec/git-core/* to bin/git?Ævar Arnfjörð Bjarmason, Mar 13, 2018
  6. Junio C HamanoMar 13, 2018
  7. Randall S. BeckerMar 13, 2018
  8. 0/3 Makefile: add a INSTALL_SYMLINKS optionÆvar Arnfjörð Bjarmason, Mar 13, 2018
  9. 1/3 Makefile: fix broken bindir_relative variableÆvar Arnfjörð Bjarmason, Mar 13, 2018
  10. 3/3 Makefile: optionally symlink libexec/git-core binaries to bin/gitÆvar Arnfjörð Bjarmason, Mar 13, 2018
  11. Johannes SixtMar 14, 2018
  12. Ævar Arnfjörð BjarmasonMar 14, 2018
  13. Linus TorvaldsMar 14, 2018
  14. Johannes SchindelinMar 15, 2018
  15. Linus TorvaldsMar 15, 2018
  16. Johannes SchindelinMar 16, 2018
  17. Ævar Arnfjörð BjarmasonMar 16, 2018
  18. Johannes SchindelinMar 19, 2018
  19. Linus TorvaldsMar 19, 2018
  20. 0/5 stop installing old libexec aliases like "git-init"Ævar Arnfjörð Bjarmason, Nov 2, 2018
  21. Junio C HamanoNov 3, 2018
  22. Ævar Arnfjörð BjarmasonNov 5, 2018
  23. Johannes SchindelinNov 12, 2018
  24. Ævar Arnfjörð BjarmasonNov 16, 2018
  25. Michael HaggertyNov 16, 2018
  26. Ævar Arnfjörð BjarmasonNov 16, 2018
  27. Jeff KingNov 17, 2018
  28. Johannes SchindelinNov 22, 2018
  29. Jeff KingNov 22, 2018
  30. Johannes SchindelinNov 23, 2018
  31. 1/5 Makefile: move long inline shell loops in "install" into helperÆvar Arnfjörð Bjarmason, Nov 2, 2018
  32. Eric SunshineNov 4, 2018
  33. Johannes SchindelinNov 12, 2018
  34. Ævar Arnfjörð BjarmasonNov 12, 2018
  35. Johannes SchindelinNov 12, 2018
  36. Ævar Arnfjörð BjarmasonNov 16, 2018
  37. 2/5 Makefile: conform some of the code to our coding standardsÆvar Arnfjörð Bjarmason, Nov 2, 2018
  38. 3/5 Makefile: stop hiding failures during "install"Ævar Arnfjörð Bjarmason, Nov 2, 2018
  39. 4/5 Makefile: add NO_INSTALL_SYMLINKS_FALLBACK switchÆvar Arnfjörð Bjarmason, Nov 2, 2018
  40. Eric SunshineNov 4, 2018
  41. 5/5 Makefile: Add a NO_INSTALL_BUILTIN_EXECDIR_ALIASES flagÆvar Arnfjörð Bjarmason, Nov 2, 2018
  42. Eric SunshineNov 4, 2018
  43. Johannes SchindelinNov 12, 2018
  44. Johannes SchindelinMar 15, 2018
  45. 2/3 Makefile: add a gitexecdir_relative variableÆvar Arnfjörð Bjarmason, Mar 13, 2018
  46. Ævar Arnfjörð BjarmasonMar 14, 2018
  47. Junio C HamanoMar 14, 2018
  48. Johannes SchindelinMar 15, 2018
  49. Duy NguyenMar 16, 2018
  50. Johannes SchindelinMar 30, 2018
  51. Martin ÅgrenMar 9, 2018
  52. Duy NguyenMar 9, 2018
  53. Junio C HamanoMar 9, 2018

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.