{"thread":{"id":"24650","subject":"Bizarro race conditions in the Git Makefile","startedAt":"2010-08-06T19:44:57Z","lastAt":"2010-08-06T21:40:49Z","messageCount":3,"participants":["Ævar Arnfjörð Bjarmason","Jonathan Nieder"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"147339","messageId":"AANLkTim2NMi3Vf-EGbFwy370q-YseQoGj=QLGMAq6N=B@mail.gmail.com","threadId":"24650","inReplyTo":null,"subject":"Bizarro race conditions in the Git Makefile","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-08-06T19:44:57Z","receivedAt":"2010-08-06T19:44:57Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"   \"Doctor, when I poke my eye like this it hurts\"\n   -\"So don't do that then\"\n\nIf you run the Git Makefile (with GNU Make 3.81) in parallel for long\nenough you'll get some interesting breakages. Those interested in\npoking their eyes can try:\n\n    while nice -n 30 make -j 15 clean all CFLAGS=-O0 CC=gcc; do 1; done\n\nThese suggest that we have some bugs in our Makefile dependencies, but\nsince I haven't found what they are I'm just going to post the\nsymptoms.\n\nThe first example is the generation of the perl.mak from the\ntop-level. It seems that if you run make with -j $n the perl.mak will\nget generated $n times:\n\n        LINK git-fast-import\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n        LINK git-imap-send\n    Writing perl.mak for Git\n        LINK git-shell\n        LINK git-show-index\n        LINK git-upload-pack\n    Writing perl.mak for Git\n    [...]\n\nThis can lead to an error where they trip over each other:\n\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n        AR libgit.a\n        AR xdiff/lib.a\n        LINK git-fast-import\n        LINK git-imap-send\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n    rename MakeMaker.tmp => perl.mak: No such file or directory at\n/usr/share/perl/5.10/ExtUtils/MakeMaker.pm line 1004.\n    make[2]: perl.mak: No such file or directory\n    make[2]: *** No rule to make target `perl.mak'.  Stop.\n    make[1]: *** [instlibdir] Error 2\n    make: *** [git-svn] Error 2\n    make: *** Waiting for unfinished jobs....\n    make[2]: perl.mak: No such file or directory\n    make[2]: *** No rule to make target `perl.mak'.  Stop.\n    make[1]: *** [instlibdir] Error 2\n    make: *** [git-relink] Error 2\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n    Writing perl.mak for Git\n\nHere one make job seemingly had its MakeMaker.tmp file usurped by\nanother job.\n\nHere's another one, it seems that the builtin/help.o dependency on\ncommon-cmds.h is broken:\n\n        CC builtin/help.o\n        [...]\n        CC builtin/ls-remote.o\n    builtin/help.c:9:25: error: common-cmds.h: No such file or directory\n    builtin/help.c: In function ‘list_common_cmds_help’:\n    builtin/help.c:278: error: ‘common_cmds’ undeclared (first use in\nthis function)\n    builtin/help.c:278: error: (Each undeclared identifier is reported only once\n    builtin/help.c:278: error: for each function it appears in.)\n\nWhich is odd, because this seems OK:\n\n    builtin/help.o: common-cmds.h\n    builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \\\n        '-DGIT_HTML_PATH=\"$(htmldir_SQ)\"' \\\n        '-DGIT_MAN_PATH=\"$(mandir_SQ)\"' \\\n        '-DGIT_INFO_PATH=\"$(infodir_SQ)\"'\n\nA vaguely recall having run into some other make error, but I can't\nreproduce it now. I'll keep poking my eye and reporting results to the\nlist.\n"},{"id":"147345","messageId":"20100806211407.GB6495@burratino","threadId":"24650","inReplyTo":"AANLkTim2NMi3Vf-EGbFwy370q-YseQoGj=QLGMAq6N=B@mail.gmail.com","subject":"Re: Bizarro race conditions in the Git Makefile","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2010-08-06T21:14:07Z","receivedAt":"2010-08-06T21:14:07Z","isPatch":false,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"Ævar Arnfjörð Bjarmason wrote:\n\n> Those interested in\n> poking their eyes can try:\n>\n>     while nice -n 30 make -j 15 clean all CFLAGS=-O0 CC=gcc; do 1; done\n\nYou are asking make to simultaneously build and unbuild everything.\nIt does not really surprise me that it gets confused.\n\nDoes\n\n while\n\tnice -n 30 make -j 15 clean CFLAGS=-O0 CC=gcc &&\n\tnice -n 30 make -j 15 all CFLAGS=-O0 CC=gcc\n do\n\t:\n done\n\nbehave better?\n\n>     Writing perl.mak for Git\n>     Writing perl.mak for Git\n>     rename MakeMaker.tmp => perl.mak: No such file or directory at\n\nMaybe something like the following is needed to protect against\ninterrupted builds (not the problem you reported, but this reminds\nme).\n\ndiff --git a/perl/Makefile b/perl/Makefile\nindex 4ab21d6..eef7ee5 100644\n--- a/perl/Makefile\n+++ b/perl/Makefile\n@@ -22,20 +22,21 @@ clean:\n ifdef NO_PERL_MAKEMAKER\n instdir_SQ = $(subst ','\\'',$(prefix)/lib)\n $(makfile): ../GIT-CFLAGS Makefile\n-\techo all: private-Error.pm Git.pm > $@\n-\techo '\tmkdir -p blib/lib' >> $@\n-\techo '\t$(RM) blib/lib/Git.pm; cp Git.pm blib/lib/' >> $@\n-\techo '\t$(RM) blib/lib/Error.pm' >> $@\n+\techo all: private-Error.pm Git.pm > $@+\n+\techo '\tmkdir -p blib/lib' >> $@+\n+\techo '\t$(RM) blib/lib/Git.pm; cp Git.pm blib/lib/' >> $@+\n+\techo '\t$(RM) blib/lib/Error.pm' >> $@+\n \t'$(PERL_PATH_SQ)' -MError -e 'exit($$Error::VERSION < 0.15009)' || \\\n-\techo '\tcp private-Error.pm blib/lib/Error.pm' >> $@\n-\techo install: >> $@\n-\techo '\tmkdir -p \"$$(DESTDIR)$(instdir_SQ)\"' >> $@\n-\techo '\t$(RM) \"$$(DESTDIR)$(instdir_SQ)/Git.pm\"; cp Git.pm \"$$(DESTDIR)$(instdir_SQ)\"' >> $@\n-\techo '\t$(RM) \"$$(DESTDIR)$(instdir_SQ)/Error.pm\"' >> $@\n+\techo '\tcp private-Error.pm blib/lib/Error.pm' >> $@+\n+\techo install: >> $@+\n+\techo '\tmkdir -p \"$$(DESTDIR)$(instdir_SQ)\"' >> $@+\n+\techo '\t$(RM) \"$$(DESTDIR)$(instdir_SQ)/Git.pm\"; cp Git.pm \"$$(DESTDIR)$(instdir_SQ)\"' >> $@+\n+\techo '\t$(RM) \"$$(DESTDIR)$(instdir_SQ)/Error.pm\"' >> $@+\n \t'$(PERL_PATH_SQ)' -MError -e 'exit($$Error::VERSION < 0.15009)' || \\\n-\techo '\tcp private-Error.pm \"$$(DESTDIR)$(instdir_SQ)/Error.pm\"' >> $@\n-\techo instlibdir: >> $@\n-\techo '\techo $(instdir_SQ)' >> $@\n+\techo '\tcp private-Error.pm \"$$(DESTDIR)$(instdir_SQ)/Error.pm\"' >> $@+\n+\techo instlibdir: >> $@+\n+\techo '\techo $(instdir_SQ)' >> $@+\n+\tmv $@+ $@\n else\n $(makfile): Makefile.PL ../GIT-CFLAGS\n \t$(PERL_PATH) $< PREFIX='$(prefix_SQ)'\n-- \n"},{"id":"147352","messageId":"AANLkTin=6pmfPjmM-W4UFf1g+0Z9Rrprc0h0mThUOUFx@mail.gmail.com","threadId":"24650","inReplyTo":"20100806211407.GB6495@burratino","subject":"Re: Bizarro race conditions in the Git Makefile","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-08-06T21:40:49Z","receivedAt":"2010-08-06T21:40:49Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Fri, Aug 6, 2010 at 21:14, Jonathan Nieder <jrnieder@gmail.com> wrote:\n> Ęvar Arnfjörš Bjarmason wrote:\n>\n>> Those interested in\n>> poking their eyes can try:\n>>\n>>     while nice -n 30 make -j 15 clean all CFLAGS=-O0 CC=gcc; do 1; done\n>\n> You are asking make to simultaneously build and unbuild everything.\n> It does not really surprise me that it gets confused.\n>\n> Does\n>\n>  while\n>        nice -n 30 make -j 15 clean CFLAGS=-O0 CC=gcc &&\n>        nice -n 30 make -j 15 all CFLAGS=-O0 CC=gcc\n>  do\n>        :\n>  done\n>\n> behave better?\n\nYes that works. I thought that -j $n would mean that make would use $n\njobs to complete the first target, then move onto the next. Not\nexecute them all in paralell.\n\nThanks, and I have no idea about those Makefile/Perl changes.\n"}]}