Bizarro race conditions in the Git Makefile

   "Doctor, when I poke my eye like this it hurts"
   -"So don't do that then"

If you run the Git Makefile (with GNU Make 3.81) in parallel for long enough you'll get some interesting breakages. Those interested in poking their eyes can try:

    while nice -n 30 make -j 15 clean all CFLAGS=-O0 CC=gcc; do 1; done

These suggest that we have some bugs in our Makefile dependencies, but since I haven't found what they are I'm just going to post the symptoms.

The first example is the generation of the perl.mak from the top-level. It seems that if you run make with -j $n the perl.mak will get generated $n times:

        LINK git-fast-import
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git
        LINK git-imap-send
    Writing perl.mak for Git
        LINK git-shell
        LINK git-show-index
        LINK git-upload-pack
    Writing perl.mak for Git
    [...]

This can lead to an error where they trip over each other:

    Writing perl.mak for Git
    Writing perl.mak for Git
        AR libgit.a
        AR xdiff/lib.a
        LINK git-fast-import
        LINK git-imap-send
    Writing perl.mak for Git
    Writing perl.mak for Git
    rename MakeMaker.tmp => perl.mak: No such file or directory at
/usr/share/perl/5.10/ExtUtils/MakeMaker.pm line 1004.
    make[2]: perl.mak: No such file or directory
    make[2]: *** No rule to make target `perl.mak'.  Stop.
    make[1]: *** [instlibdir] Error 2
    make: *** [git-svn] Error 2
    make: *** Waiting for unfinished jobs....
    make[2]: perl.mak: No such file or directory
    make[2]: *** No rule to make target `perl.mak'.  Stop.
    make[1]: *** [instlibdir] Error 2
    make: *** [git-relink] Error 2
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git
    Writing perl.mak for Git

Here one make job seemingly had its MakeMaker.tmp file usurped by another job.

Here's another one, it seems that the builtin/help.o dependency on common-cmds.h is broken:

        CC builtin/help.o
        [...]
        CC builtin/ls-remote.o
    builtin/help.c:9:25: error: common-cmds.h: No such file or directory
    builtin/help.c: In function ‘list_common_cmds_help’:
    builtin/help.c:278: error: ‘common_cmds’ undeclared (first use in
this function)
    builtin/help.c:278: error: (Each undeclared identifier is reported only once
    builtin/help.c:278: error: for each function it appears in.)

Which is odd, because this seems OK:

    builtin/help.o: common-cmds.h
    builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \
        '-DGIT_HTML_PATH="$(htmldir_SQ)"' \
        '-DGIT_MAN_PATH="$(mandir_SQ)"' \
        '-DGIT_INFO_PATH="$(infodir_SQ)"'

A vaguely recall having run into some other make error, but I can't reproduce it now. I'll keep poking my eye and reporting results to the list.

Ævar Arnfjörð Bjarmason wrote:

You are asking make to simultaneously build and unbuild everything. It does not really surprise me that it gets confused.

Does

 while
	nice -n 30 make -j 15 clean CFLAGS=-O0 CC=gcc &&
	nice -n 30 make -j 15 all CFLAGS=-O0 CC=gcc
 do
	:
 done

behave better?

Maybe something like the following is needed to protect against interrupted builds (not the problem you reported, but this reminds me).

On Fri, Aug 6, 2010 at 21:14, Jonathan Nieder <jrnieder@gmail.com> wrote:

Yes that works. I thought that -j $n would mean that make would use $n jobs to complete the first target, then move onto the next. Not execute them all in paralell.

Thanks, and I have no idea about those Makefile/Perl changes.