threads / patch / 4510

patchauto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

Subject: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

## tl;dr

17 messages between Jun 14, 2006 and Jun 18, 2006. Diffs are folded; open one to read it.

replies: 16people: 7as markdown or json

Yakov Lerner· Jun 14, 2006, 19:26 UTC · lore

Many times, I mistakenly used 'make prefix=... install' where prefix value was different from prefix value during build. This resulted in broken install. This patch adds auto-detection of $prefix change to the Makefile. This results in correct install whenever prefix is changed.

Signed-off-by: Yakov Lerner <iler.ml@gmail.com>
---
 Makefile |   29 ++++++++++++++++++++++-------
 1 files changed, 22 insertions(+), 7 deletions(-)
Show changes to Makefile +22 −7
diff --git a/Makefile b/Makefile
index 2a1e639..015c9b2 100644
--- a/Makefile
+++ b/Makefile
@@ -464,6 +464,7 @@ DESTDIR_SQ = $(subst ','\'',$(DESTDIR))
 bindir_SQ = $(subst ','\'',$(bindir))
 gitexecdir_SQ = $(subst ','\'',$(gitexecdir))
 template_dir_SQ = $(subst ','\'',$(template_dir))
+prefix_SQ = $(subst ','\'',$(prefix))
 
 SHELL_PATH_SQ = $(subst ','\'',$(SHELL_PATH))
 PERL_PATH_SQ = $(subst ','\'',$(PERL_PATH))
@@ -484,7 +485,7 @@ all:
 strip: $(PROGRAMS) git$X
 	$(STRIP) $(STRIP_OPTS) $(PROGRAMS) git$X
 
-git$X: git.c common-cmds.h $(BUILTIN_OBJS) $(GITLIBS)
+git$X: git.c common-cmds.h $(BUILTIN_OBJS) $(GITLIBS) .git.prefix
 	$(CC) -DGIT_VERSION='"$(GIT_VERSION)"' \
 		$(ALL_CFLAGS) -o $@ $(filter %.c,$^) \
 		$(BUILTIN_OBJS) $(ALL_LDFLAGS) $(LIBS)
@@ -516,7 +517,7 @@ common-cmds.h: Documentation/git-*.txt
 	chmod +x $@+
 	mv $@+ $@
 
-$(patsubst %.py,%,$(SCRIPT_PYTHON)) : % : %.py
+$(patsubst %.py,%,$(SCRIPT_PYTHON)) : % : %.py .git.prefix
 	rm -f $@ $@+
 	sed -e '1s|#!.*python|#!$(PYTHON_PATH_SQ)|' \
 	    -e 's|@@GIT_PYTHON_PATH@@|$(GIT_PYTHON_DIR_SQ)|g' \
@@ -540,19 +541,19 @@ git$X git.spec \
 	$(patsubst %.py,%,$(SCRIPT_PYTHON)) \
 	: GIT-VERSION-FILE
 
-%.o: %.c
+%.o: %.c .git.prefix
 	$(CC) -o $*.o -c $(ALL_CFLAGS) $<
 %.o: %.S
 	$(CC) -o $*.o -c $(ALL_CFLAGS) $<
 
-exec_cmd.o: exec_cmd.c
+exec_cmd.o: exec_cmd.c .git.prefix
 	$(CC) -o $*.o -c $(ALL_CFLAGS) '-DGIT_EXEC_PATH="$(gitexecdir_SQ)"' $<
 
-http.o: http.c
+http.o: http.c .git.prefix
 	$(CC) -o $*.o -c $(ALL_CFLAGS) -DGIT_USER_AGENT='"git/$(GIT_VERSION)"' $<
 
 ifdef NO_EXPAT
-http-fetch.o: http-fetch.c http.h
+http-fetch.o: http-fetch.c http.h .git.prefix
 	$(CC) -o $*.o -c $(ALL_CFLAGS) -DNO_EXPAT $<
 endif
 
@@ -609,6 +610,14 @@ tags:
 	rm -f tags
 	find . -name '*.[hcS]' -print | xargs ctags -a
 
+### Detect prefix changes
+.git.prefix: .FORCE-git.prefix
+	@PREFIXES='$(bindir_SQ):$(gitexecdir_SQ):$(template_dir_SQ):$(prefix_SQ)';\
+	    if test x"$$PREFIXES" != x"`cat .git.prefix 2>/dev/null`" ; then \
+		echo 1>&2 "    * prefix changed"; \
+		echo "$$PREFIXES" >.git.prefix; \
+            fi
+
 ### Testing rules
 
 # GNU make supports exporting all variables by "export" without parameters.
@@ -632,6 +641,12 @@ test-dump-cache-tree$X: dump-cache-tree.
 check:
 	for i in *.c; do sparse $(ALL_CFLAGS) $(SPARSE_FLAGS) $$i || exit; done
 
+test-prefix-change:
+	mkdir -p "`pwd`/tmp1" "`pwd`/tmp2"
+	$(MAKE) clean install prefix="`pwd`/tmp1"
+	$(MAKE) install prefix="`pwd`/tmp2"
+	@grep -r "`pwd`/tmp1" "`pwd`/tmp2" >/dev/null; if test $$? = 0 ; then\
+	    echo Error, test failed; exit 1; else echo Ok, test passed; fi
 
 
 ### Installation rules
@@ -714,7 +729,7 @@ clean:
 	rm -f GIT-VERSION-FILE
 
 .PHONY: all install clean strip
-.PHONY: .FORCE-GIT-VERSION-FILE TAGS tags
+.PHONY: .FORCE-GIT-VERSION-FILE TAGS tags .FORCE-git.prefix
 
 ### Check documentation
 #
-- 
1.4.0
Junio C Hamano· Jun 14, 2006, 20:04 UTC · re: Yakov Lerner · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

Yakov Lerner <iler.ml@gmail.com> writes:
Show 6 quoted lines
> Many times, I mistakenly used 'make prefix=... install' where prefix value
> was different from prefix value during build. This resulted in broken
> install. This patch adds auto-detection of $prefix change to the Makefile.
> This results in correct install whenever prefix is changed.
>
> Signed-off-by: Yakov Lerner <iler.ml@gmail.com>

I do not mind this per se, and probably even agree that this is an improvement compared to the current state of affairs, but a few points:

 - please make sure you clean that state file in "make clean";
 - we may want to make the state file a bit more visible (IOW, I
   somewhat do mind the name being dot-git-dot-prefix).
 - we might want to later (or at the same time as this patch)
   do "consistent set of compilation flags" (e.g. run early
   part of compilation with openssl SHA-1 implementation,
   interrupt it and build and link the rest with mozilla SHA-1
   implementation -- then you will get a nonsense binary without
   linker errors).  It might make sense to prepare this
   mechanism so we could reuse it for that purpose.
Yakov Lerner· Jun 14, 2006, 20:30 UTC · re: Junio C Hamano · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

On 6/14/06, Junio C Hamano <junkio@cox.net> wrote:
Show 14 quoted lines
> Yakov Lerner <iler.ml@gmail.com> writes:
>
> > Many times, I mistakenly used 'make prefix=... install' where prefix value
> > was different from prefix value during build. This resulted in broken
> > install. This patch adds auto-detection of $prefix change to the Makefile.
> > This results in correct install whenever prefix is changed.
> >
> > Signed-off-by: Yakov Lerner <iler.ml@gmail.com>
>
> I do not mind this per se, and probably even agree that this is
> an improvement compared to the current state of affairs, but a few
> points:
>
>  - please make sure you clean that state file in "make clean";
done
>  - we may want to make the state file a bit more visible (IOW, I
>    somewhat do mind the name being dot-git-dot-prefix).
I renamed .git.prefix to GIT-PREFIX. Is this ok.
Show 7 quoted lines
>  - we might want to later (or at the same time as this patch)
>    do "consistent set of compilation flags" (e.g. run early
>    part of compilation with openssl SHA-1 implementation,
>    interrupt it and build and link the rest with mozilla SHA-1
>    implementation -- then you will get a nonsense binary without
>    linker errors).  It might make sense to prepare this
>    mechanism so we could reuse it for that purpose.

Do you think two separate GIT-PREFIX and GIT-BUILD-FLAGS are needed, or just once GIT-BUILD-FLAGS will do, which will include prefixes (as passed with -D... to cc) ?

I think single GIT-BUILD-FLAGS is enough, which will cover prefixes, too. Is this OK ?

BTW, I think it's useful to add Makefile itself as prerequisite for all *.o, so change in Makefile will cause recompilations. Shall I include this into this patch, too ?

Yakov
Junio C Hamano· Jun 14, 2006, 21:32 UTC · re: Yakov Lerner · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

"Yakov Lerner" <iler.ml@gmail.com> writes:
> I think single GIT-BUILD-FLAGS
> is enough, which will cover prefixes, too. Is this OK ?

Yes, it was what I was getting at. I think a single GIT-BUILD-FLAGS (or whatever name the list can fight over while I am away) is preferred.

> BTW, I think it's useful to add Makefile itself as prerequisite for all *.o,
> so change in Makefile will cause recompilations. Shall I include this
> into this patch, too ?

I've thought about it but in practice this would make things more inconvenient for developers without much gain, so I'd leave it out.

Yakov Lerner· Jun 14, 2006, 21:38 UTC · re: Junio C Hamano · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

On 6/15/06, Junio C Hamano <junkio@cox.net> wrote:
Show 8 quoted lines
> "Yakov Lerner" <iler.ml@gmail.com> writes:
>
> > I think single GIT-BUILD-FLAGS
> > is enough, which will cover prefixes, too. Is this OK ?
>
> Yes, it was what I was getting at.  I think a single
> GIT-BUILD-FLAGS (or whatever name the list can fight over while
> I am away) is preferred.

Either GIT-CFLAGS or GIT-BUILD-FLAGS, whichever is shorter :-)

Yakov
Matthias Lederhofer· Jun 15, 2006, 09:26 UTC · re: Yakov Lerner · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

> Either GIT-CFLAGS or GIT-BUILD-FLAGS,
> whichever is shorter :-)

I would not take GIT-CFLAGS because there is an environment variable used by make named CFLAGS and means something else.

What is the target test-prefix-change good for? Should it really be included?

Yakov Lerner· Jun 15, 2006, 11:11 UTC · re: Matthias Lederhofer · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

On 6/15/06, Matthias Lederhofer <matled@gmx.net> wrote:
Show 5 quoted lines
> > Either GIT-CFLAGS or GIT-BUILD-FLAGS,
> > whichever is shorter :-)
>
> I would not take GIT-CFLAGS because there is an environment variable
> used by make named CFLAGS and means something else.

What is your preference for this filename ? Is GIT-BUILD-FLAGS better ? GIT-FLAGS ? Other ?

GIT-CFLAGS is filename here, whereas $(CFLAGS) is variable. Maybe writing it the filename ./GIT-CFLAGS would emphasize betterthat it's not make variable, but a filename ?

> What is the target test-prefix-change good for? Should it really be
> included?

The purpose of 'test-prefix-change' is to check that there are no missing GIT-FLAGS as prerequisite which would result in broken install if prefix changed between 'make' and 'make install'. The 'make test-prefix-change' would catch it if GIT-FLAGS happened to be missing in some places that if substantially dependent of $prefix.

Yakov
Petr Baudis· Jun 18, 2006, 13:31 UTC · re: Junio C Hamano · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

Dear diary, on Wed, Jun 14, 2006 at 10:04:43PM CEST, I got a letter where Junio C Hamano <junkio@cox.net> said that...

>  - we may want to make the state file a bit more visible (IOW, I
>    somewhat do mind the name being dot-git-dot-prefix).

What is the point? It is just a bit of internal build system state made persistent and shouldn't be interesting for the user, so why give it extra publicity in the tree?

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
A person is just about as big as the things that make them angry.
Santi· Jun 15, 2006, 11:26 UTC · re: Yakov Lerner · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

Wed, 14 Jun 2006 22:26 +0300, Yakov Lerner <iler.ml@gmail.com>:
> Many times, I mistakenly used 'make prefix=... install' where prefix value
> was different from prefix value during build. This resulted in broken
> install. This patch adds auto-detection of $prefix change to the Makefile.
> This results in correct install whenever prefix is changed.

I do this each time I install packages from source. I keep them with "stow" and the usual sequence is:

make prefix=/home/santi/usr make install prefix=/home/santi/usr/stow/git cd /home/santi/usr/stow/ stow -v git

so with this auto-detection I'll compile some programs twice.
Santi
Yakov Lerner· Jun 15, 2006, 11:40 UTC · re: Santi · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

On 6/15/06, Santi <sbejar@gmail.com> wrote:
Show 15 quoted lines
> Wed, 14 Jun 2006 22:26 +0300, Yakov Lerner <iler.ml@gmail.com>:
> > Many times, I mistakenly used 'make prefix=... install' where prefix value
> > was different from prefix value during build. This resulted in broken
> > install. This patch adds auto-detection of $prefix change to the Makefile.
> > This results in correct install whenever prefix is changed.
>
> I do this each time I install packages from source. I keep them with
> "stow" and the usual sequence is:
>
> make prefix=/home/santi/usr
> make install prefix=/home/santi/usr/stow/git
> cd /home/santi/usr/stow/
> stow -v git
>
> so with this auto-detection I'll compile some programs twice.

I'm not familiar with stow. Does stow create some kind of symlinks from /home/santi/usr/stow/git to home/santi/usr ? If so, why can't you use prefix=/home/santi/usr/stow/git both in 'make' and in 'make install' ? Would this work ?

BTW, is it possible to have git use argv[0] to automatically determine the executable_dir without compiled-in paths ?

Yakov
Santi· Jun 15, 2006, 12:49 UTC · re: Yakov Lerner · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

2006/6/15, Yakov Lerner <iler.ml@gmail.com>:
Show 19 quoted lines
> On 6/15/06, Santi <sbejar@gmail.com> wrote:
> > Wed, 14 Jun 2006 22:26 +0300, Yakov Lerner <iler.ml@gmail.com>:
> > > Many times, I mistakenly used 'make prefix=... install' where prefix value
> > > was different from prefix value during build. This resulted in broken
> > > install. This patch adds auto-detection of $prefix change to the Makefile.
> > > This results in correct install whenever prefix is changed.
> >
> > I do this each time I install packages from source. I keep them with
> > "stow" and the usual sequence is:
> >
> > make prefix=/home/santi/usr
> > make install prefix=/home/santi/usr/stow/git
> > cd /home/santi/usr/stow/
> > stow -v git
> >
> > so with this auto-detection I'll compile some programs twice.
>
> I'm not familiar with stow. Does stow create some kind of symlinks from
> /home/santi/usr/stow/git to home/santi/usr ?
It makes links from .../usr/ to .../usr/stow/git/, for expample:

$ls -l ~/usr/bin/git lrwxrwxrwx 1 santi santi 19 2006-06-01 09:42 /home/santi/usr/bin/git -> ../stow/git/bin/git*

> If so, why can't you
> use prefix=/home/santi/usr/stow/git both in 'make' and in 'make install' ?
> Would this work ?

For the few tests I've made it does works, but it is not the recommended method from the stow developers, and I suppose they know better than me.

Show 6 quoted lines
>
> BTW, is it possible to have git use argv[0] to automatically determine
> the executable_dir without compiled-in paths ?
>
> Yakov
>
Alexander Litvinov· Jun 15, 2006, 13:00 UTC · re: Santi · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

Show 7 quoted lines
> > If so, why can't you
> > use prefix=/home/santi/usr/stow/git both in 'make' and in 'make install'
> > ? Would this work ?
>
> For the few tests I've made it does works, but it is not the
> recommended method from the stow developers, and I suppose they know
> better than me.

I always use make prefix=/usr/local/stow/git-bla-bla and it works. The only thing that make break is the case when you install library into such prefix and it store it in the pkgconfig file. And when you will compile application it will be build with library paths set to /usr/local/stow/lib/libxyz and not to /usr/local/lib/libxyz.

But I dont make such big dependences from sources so it works.
Junio C Hamano· Jun 17, 2006, 05:28 UTC · re: Yakov Lerner · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

"Yakov Lerner" <iler.ml@gmail.com> writes:
> BTW, is it possible to have git use argv[0] to automatically determine
> the executable_dir without compiled-in paths ?
It is possible and I think we did that some time ago.

We fixed it since then, so that you can migrate away from assuming bindir==gitexecdir.

Junio C Hamano· Jun 17, 2006, 05:26 UTC · re: Santi · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

Santi <sbejar@gmail.com> writes:
Show 15 quoted lines
> Wed, 14 Jun 2006 22:26 +0300, Yakov Lerner <iler.ml@gmail.com>:
>> Many times, I mistakenly used 'make prefix=... install' where prefix value
>> was different from prefix value during build. This resulted in broken
>> install. This patch adds auto-detection of $prefix change to the Makefile.
>> This results in correct install whenever prefix is changed.
>
> I do this each time I install packages from source. I keep them with
> "stow" and the usual sequence is:
>
> make prefix=/home/santi/usr
> make install prefix=/home/santi/usr/stow/git
> cd /home/santi/usr/stow/
> stow -v git
>
> so with this auto-detection I'll compile some programs twice.

I do not know what "stow" is about, but if it is to allow you to run make-install to install things in somewhere else, examine the result, and then move the result to the real location (implying that you should be able to nuke the "somewhere else" after you have done so), with the patch, the above sequence would install the binaries pointing at a wrong directory, because the second compilation would make them point at the temporary installation directory ~/usr/stow/git, not the final location ~/usr/.

Karl Hasselström· Jun 18, 2006, 11:24 UTC · re: Junio C Hamano · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

On 2006-06-16 22:26:38 -0700, Junio C Hamano wrote:
Show 8 quoted lines
> I do not know what "stow" is about, but if it is to allow you to run
> make-install to install things in somewhere else, examine the
> result, and then move the result to the real location (implying that
> you should be able to nuke the "somewhere else" after you have done
> so), with the patch, the above sequence would install the binaries
> pointing at a wrong directory, because the second compilation would
> make them point at the temporary installation directory
> ~/usr/stow/git, not the final location ~/usr/.

GNU stow doesn't move installed programs, it just maintains symlinks to them. You install programs under /usr/local/stow/foo-4.7.11, and stow sets up symlinks to them under /usr/local. (So for example, /usr/local/bin/foo would be a symlink to /usr/local/stow/foo-4.7.11/bin/foo.) This gives you the ability to nuke an installed program cleanly. And it just works, pathwise, since the program remains in its original location.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle
Junio C Hamano· Jun 18, 2006, 11:47 UTC · re: Karl Hasselström · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

Karl Hasselström <kha@treskal.com> writes:
Show 7 quoted lines
> GNU stow doesn't move installed programs, it just maintains symlinks
> to them. You install programs under /usr/local/stow/foo-4.7.11, and
> stow sets up symlinks to them under /usr/local. (So for example,
> /usr/local/bin/foo would be a symlink to
> /usr/local/stow/foo-4.7.11/bin/foo.) This gives you the ability to
> nuke an installed program cleanly. And it just works, pathwise, since
> the program remains in its original location.
Thanks for the explanation.

If that's the case, I think it makes the original problem Santi brought up a non-issue. In this sequence:

        make prefix=/home/santi/usr
        make install prefix=/home/santi/usr/stow/git
        cd /home/santi/usr/stow/
        stow -v git

the building phase could have used the same prefix as the install phase uses, and git can find its subprograms in gitexecdir (= ~/usr/stow/git/bin) just fine. It probably is even slightly more efficient since it does not have to go through the symlink stow installs.

Karl Hasselström· Jun 18, 2006, 14:44 UTC · re: Junio C Hamano · lore

Re: [PATCH] auto-detect changed $prefix in Makefile and properly rebuild to avoid broken install

On 2006-06-18 04:47:19 -0700, Junio C Hamano wrote:
Show 15 quoted lines
> Thanks for the explanation.
>
> If that's the case, I think it makes the original problem Santi
> brought up a non-issue. In this sequence:
>
>         make prefix=/home/santi/usr
>         make install prefix=/home/santi/usr/stow/git
>         cd /home/santi/usr/stow/
>         stow -v git
>
> the building phase could have used the same prefix as the install
> phase uses, and git can find its subprograms in gitexecdir (=
> ~/usr/stow/git/bin) just fine. It probably is even slightly more
> efficient since it does not have to go through the symlink stow
> installs.
Yes, exactly. I've always built git like this:
  $ make prefix=/usr/local/stow/git
  $ sudo make prefix=/usr/local/stow/git install
  $ cd /usr/local/stow
  $ sudo stow git

It works for all other programs I've tried too (most of which only require me to specify the prefix once, with ./configure --prefix=...). The programs never need to know about the symlinks; they're only there for when other programs need to access them (via PATH, etc.).

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

← back to recent threads