git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 1/1] exec_cmd: RUNTIME_PREFIX on some POSIX systems

From
Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date
Nov 20, 2017, 21:00 UTC
Message-ID
<87lgj0wtr9.fsf@evledraar.booking.com>
In-Reply-To
<20171119173141.4896-2-dnj@google.com>
On Sun, Nov 19 2017, Dan Jacques jotted:
> [...]

Firstly the promise of this is very neat. I'm happy to offer any help I can give.

Show 20 quoted lines
> Enable Git to resolve its own binary location using a variety of
> OS-specific and generic methods, including:
>
> - procfs via "/proc/self/exe" (Linux)
> - _NSGetExecutablePath (Darwin)
> - KERN_PROC_PATHNAME sysctl on BSDs.
> - argv0, if absolute (all, including Windows).
>
> This is used to enable RUNTIME_PREFIX support for non-Windows systems,
> notably Linux and Darwin. When configured with RUNTIME_PREFIX, Git will
> do a best-effort resolution of its executable path and automatically use
> this as its "exec_path" for relative helper and data lookups, unless
> explicitly overridden.
>
> Git's PERL tooling now responds to RUNTIME_PREFIX_PERL. When configured,
> Git's generated PERL scripts resolve the Git library location relative to
> their runtime paths instead of hard-coding them. Structural changes
> were made to Makefile to support selective PERL header generation.
>
> Small incidental formatting cleanup of "exec_cmd.c".
Show 43 quoted lines
> +$(SCRIPT_PERL_GEN): % : %.perl perl/perl.mak GIT-PERL-DEFINES GIT-PERL-HEADER GIT-VERSION-FILE
>  	$(QUIET_GEN)$(RM) $@ $@+ && \
>  	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
>  	INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \
>  	INSTLIBDIR="$$INSTLIBDIR$${INSTLIBDIR_EXTRA:+:$$INSTLIBDIR_EXTRA}" && \
>  	sed -e '1{' \
>  	    -e '	s|#!.*perl|#!$(PERL_PATH_SQ)|' \
> -	    -e '	h' \
> -	    -e '	s=.*=use lib (split(/$(pathsep)/, $$ENV{GITPERLLIB} || "'"$$INSTLIBDIR"'"));=' \
> -	    -e '	H' \
> -	    -e '	x' \
> +	    -e '	rGIT-PERL-HEADER' \
> +	    -e '	G' \
>  	    -e '}' \
>  	    -e 's/@@GIT_VERSION@@/$(GIT_VERSION)/g' \
>  	    $< >$@+ && \
> @@ -1986,6 +2028,29 @@ GIT-PERL-DEFINES: FORCE
>  		echo "$$FLAGS" >$@; \
>  	    fi
>
> +GIT-PERL-HEADER: perl/perl.mak GIT-PERL-DEFINES FORCE
> +ifndef RUNTIME_PREFIX_PERL
> +	# Hardcode the runtime path.
> +	INSTLIBDIR=`MAKEFLAGS= $(MAKE) -C perl -s --no-print-directory instlibdir` && \
> +	INSTLIBDIR_EXTRA='$(PERLLIB_EXTRA_SQ)' && \
> +	echo \
> +	  'use lib (split(/$(pathsep)/, $$ENV{GITPERLLIB} || "'"$$INSTLIBDIR"'"));' \
> +	  >$@
> +else
> +	# Probe the runtime path relative to the PERL script. RUNTIME_PREFIX_PERL
> +	# automatically sets NO_PERL_MAKEMAKER, causing PERL scripts to be installed
> +	# to "$(prefix)/lib" (see "perl/Makefile"). This expectation is hard-coded
> +	# into the generated code below.
> +	GITEXECDIR='$(gitexecdir_SQ)' && \
> +	echo \
> +	  'sub _get_git_lib{'\
> +	  'use FindBin;'\
> +	  '(my $$p=$$FindBin::Bin)=~s=/'$${GITEXECDIR}'$$==;'\
> +		'return File::Spec->catdir($$p,"'"lib"'");' \
> +	  '};' \
> +	  'use lib (split(/$(pathsep)/, $$ENV{GITPERLLIB}||_get_git_lib()));'\
> +	  >$@
> +endif
If you run run:
    make -j8 prefix=/tmp/git RUNTIME_PREFIX=YesPlease RUNTIME_PREFIX_PERL= CFLAGS="-O0 -g" all install
And then:
    make -j8 prefix=/tmp/git RUNTIME_PREFIX=YesPlease RUNTIME_PREFIX_PERL=YesPlease CFLAGS="-O0 -g" all install
You end up with this:
    $ tree /tmp/git/{lib,share/perl}
    /tmp/git/lib
    └── x86_64-linux-gnu
        └── perl
            └── 5.26.1
                ├── auto
                │   └── Git
                └── perllocal.pod
    /tmp/git/share/perl
    └── 5.26.1
        [...]
        └── Git.pm

You need to bust the perl/PM.stamp cache as a function of your RUNTIME_PREFIX_PERL variable (or NO_PERL_MAKEMAKER).

Other than that, is this whole NO_PERL_MAKEMAKER workaround just because you couldn't figure out what the target RELPERLPATH is in $prefix/$RELPERLPATH, which in this case is share/perl/5.26.1 ?

I don't remember offhand how to extract that, but htis is built into perl itself, see e.g.:

    $ PERL5LIB= /usr/bin/perl -E 'say for grep { m[share|lib] } @INC'
    /usr/local/lib/x86_64-linux-gnu/perl/5.26.1
    /usr/local/share/perl/5.26.1
    /usr/lib/x86_64-linux-gnu/perl5/5.26
    /usr/share/perl5
    /usr/lib/x86_64-linux-gnu/perl/5.26
    /usr/share/perl/5.26
    /usr/local/lib/site_perl
    /usr/lib/x86_64-linux-gnu/perl-base

So it's in Config.pm somewhere IIRC. But your patch doesn't discuss why you toggled NO_PERL_MAKEMAKER, is it purely to work around this issue, and if we had some aesy way to figure out the target $RELPERLPATH we wouldn't need to do that?

Seems a bit of baby & bathwater there :)
Aside from that:
1. The regex match you're doing to munge the dir could be done as a
   catdir($orig, '..', '..', 'lib'), that doesn't work as discussed
   above, but *might* be more portable. I say might because I don't know
   if the path string is always normalized to be unix-like, but if not
   this won't work e.g on Windows where it'll have \ not /.
2. You are 'use'-ing FindBin there unconditionally (use is not function
   local in perl), and implicitly assuming it loads File::Spec.
   Ignoring the NO_PERL_MAKEMAKER caveats above I'd suggest something
   like this:
       #!/usr/bin/perl
       # BEGIN RUNTIME_PREFIX_PERL=YesPlease generated code.
       #
       # This finds our Git::* libraries at relative locations.
       BEGIN {
           use lib split /:/,
           (
               $ENV{GITPERLLIB}
               ||
               do {
                   require FindBin;
                   require File::Spec;
                   File::Spec->catdir($FindBin::Bin, '..', '..', 'lib');
               }
           );
       }
       # END RUNTIME_PREFIX_PERL=YesPlease generated code.
       # Copyright (C) 2006, Eric Wong <normalperson@yhbt.net>
       # License: GPL v2 or later
   It's also nice to have some whitespace / comments to note that this
   is generated code.
 3. I may be squinting at this wrong but it seems to me that between
    your v1 and v2 reading GITPERLLIB here no longer makes any sense at
    all. You used to set it in git itself, now it takes priority but
    nothing sets it, presumably you'd have some external wrapper script
    that'll set it?
    Now if I compile with RUNTIME_PREFIX=YesPlease I get magic
    auto-discovery of C program paths, right? But it'll still fallback
    to the system perl libs (if any) unless
    RUNTIME_PREFIX_PERL=YesPlease is set. Shouldn't we just make
    RUNTIME_PREFIX=YesPlease Just Work for everything?
Previous: Junio C HamanoNext: Ævar Arnfjörð Bjarmason
Message 4 of 10 in “exec_cmd: RUNTIME_PREFIX on some POSIX systems”
  1. 0/1 exec_cmd: RUNTIME_PREFIX on some POSIX systemsDan Jacques, Nov 19, 2017
  2. 1/1 exec_cmd: RUNTIME_PREFIX on some POSIX systemsDan Jacques, Nov 19, 2017
  3. Junio C HamanoNov 20, 2017
  4. Ævar Arnfjörð BjarmasonNov 20, 2017
  5. Ævar Arnfjörð BjarmasonNov 20, 2017
  6. Re [PATCH 1/1] exec_cmd: RUNTIME_PREFIX on some POSIX systemsDan Jacques, Nov 21, 2017
  7. Ævar Arnfjörð BjarmasonNov 22, 2017
  8. Re [PATCH 1/1] exec_cmd: RUNTIME_PREFIX on some POSIX systemsDan Jacques, Nov 22, 2017
  9. Re(2): [PATCH 1/1] exec_cmd: RUNTIME_PREFIX on some POSIX systemsDan Jacques, Nov 20, 2017
  10. Ævar Arnfjörð BjarmasonNov 20, 2017

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.