{"thread":{"id":"36803","subject":"[PATCH] Makefile: don't hardcode HEAD in dist target","startedAt":"2014-05-31T20:25:10Z","lastAt":"2014-06-02T20:27:14Z","messageCount":4,"participants":["Dennis Kaarsemaker","Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"243081","messageId":"20140531202507.GA9101@spirit","threadId":"36803","inReplyTo":null,"subject":"[PATCH] Makefile: don't hardcode HEAD in dist target","fromName":"Dennis Kaarsemaker","fromEmail":"dennis@kaarsemaker.net","sentAt":"2014-05-31T20:25:10Z","receivedAt":"2014-05-31T20:25:10Z","isPatch":true,"sender":{"key":"dennis@kaarsemaker.net","avatar":"https://avatars.githubusercontent.com/u/200649?v=4"},"body":"Instead of calling git-archive HEAD^{tree}, use $(GIT_VERSION)^{tree}.\nThis makes sure the archive name and contents are consistent, if HEAD\nhas moved, but GIT-VERSION-FILE hasn't been regenerated yet.\n\nSigned-off-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>\n---\nI have a somewhat odd setup in which I share a .git between multiple\ncheckouts for automated builds. To minimize locking time for parallel\nbuilds with different options, there's only a lock around checkout and\nrunning git describe $commit > version, the builds themselves run in\nparallel.\n\nThis works just fine except during 'make dist', which is hardcoded to\npackage up HEAD, but that's not always the commit that is actually\nchecked out, another process may have checked out something else.\n\nI realize this setup is somewhat strange, but the only change necessary\nto make this work is this one-line patch, so I hope that's acceptable.\n\n Makefile | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/Makefile b/Makefile\nindex a53f3a8..63d9bac 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -2433,7 +2433,7 @@ git.spec: git.spec.in GIT-VERSION-FILE\n GIT_TARNAME = git-$(GIT_VERSION)\n dist: git.spec git-archive$(X) configure\n \t./git-archive --format=tar \\\n-\t\t--prefix=$(GIT_TARNAME)/ HEAD^{tree} > $(GIT_TARNAME).tar\n+\t\t--prefix=$(GIT_TARNAME)/ $(GIT_VERSION)^{tree} > $(GIT_TARNAME).tar\n \t@mkdir -p $(GIT_TARNAME)\n \t@cp git.spec configure $(GIT_TARNAME)\n \t@echo $(GIT_VERSION) > $(GIT_TARNAME)/version\n-- \n2.0.0-538-ga6b2d95\n"},{"id":"243144","messageId":"xmqq4n035ej3.fsf@gitster.dls.corp.google.com","threadId":"36803","inReplyTo":"20140531202507.GA9101@spirit","subject":"Re: [PATCH] Makefile: don't hardcode HEAD in dist target","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-06-02T18:53:36Z","receivedAt":"2014-06-02T18:53:36Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Dennis Kaarsemaker <dennis@kaarsemaker.net> writes:\n\n> Instead of calling git-archive HEAD^{tree}, use $(GIT_VERSION)^{tree}.\n> This makes sure the archive name and contents are consistent, if HEAD\n> has moved, but GIT-VERSION-FILE hasn't been regenerated yet.\n>\n> Signed-off-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>\n> ---\n> I have a somewhat odd setup in which I share a .git between multiple\n> checkouts for automated builds. To minimize locking time for parallel\n> builds with different options, there's only a lock around checkout and\n> running git describe $commit > version, the builds themselves run in\n> parallel.\n>\n> This works just fine except during 'make dist', which is hardcoded to\n> package up HEAD, but that's not always the commit that is actually\n> checked out, another process may have checked out something else.\n>\n> I realize this setup is somewhat strange, but the only change necessary\n> to make this work is this one-line patch, so I hope that's acceptable.\n\nThe \"dist\" target happens to do a clean checkout to a temporary\ndirectory with \"git archive\" and then muck with its contents before\ntarring up the result, so moving HEAD around may appear to work for\nthis particular target, but htmldocs/manpages targets use what is in\nthe current checkout of Documentation/ area.  Turning the HEAD^{tree}\ninto $(GIT_VERSION)^{tree} would make the inconsistency between the\ntwo worse, wouldn't it?\n\nIf we were to change the \"dist\" rules, we may want to go in the\nopposite direction.  Instead of using \"git archive\" to make a\ntemporary copy of a directory from a commit, make such a temporary\ncopy from the contents of the current working tree (or the index).\nAnd then tar-up a result after adding configure, version etc. to it.\nThat would mean what will be in the tarball can be different from\neven HEAD, which is more consistent with the way how documentation\ntarballs are made.\n\nAlternatively, you can update the dist-doc rules to make a temporary\ncopy of documentation area and generate the documentation tarballs\nout of a pristine source of a known version---which would also make\nthese two consistent.  I am not sure which one is more preferrable,\nthough.\n\nWhy are you sharing the .git/HEAD across multiple checkouts in the\nfirst place?  If they are to build all different versions, surely\nthese working trees are derived from different commits, no?  Have\nyou considered using contrib/workdir/git-new-workdir, perhaps?\n\nI dunno.\n\n>  Makefile | 2 +-\n>  1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/Makefile b/Makefile\n> index a53f3a8..63d9bac 100644\n> --- a/Makefile\n> +++ b/Makefile\n> @@ -2433,7 +2433,7 @@ git.spec: git.spec.in GIT-VERSION-FILE\n>  GIT_TARNAME = git-$(GIT_VERSION)\n>  dist: git.spec git-archive$(X) configure\n>  \t./git-archive --format=tar \\\n> -\t\t--prefix=$(GIT_TARNAME)/ HEAD^{tree} > $(GIT_TARNAME).tar\n> +\t\t--prefix=$(GIT_TARNAME)/ $(GIT_VERSION)^{tree} > $(GIT_TARNAME).tar\n>  \t@mkdir -p $(GIT_TARNAME)\n>  \t@cp git.spec configure $(GIT_TARNAME)\n>  \t@echo $(GIT_VERSION) > $(GIT_TARNAME)/version\n"},{"id":"243150","messageId":"20140602193419.GA10198@spirit","threadId":"36803","inReplyTo":"xmqq4n035ej3.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH] Makefile: don't hardcode HEAD in dist target","fromName":"Dennis Kaarsemaker","fromEmail":"dennis@kaarsemaker.net","sentAt":"2014-06-02T19:34:20Z","receivedAt":"2014-06-02T19:34:20Z","isPatch":true,"sender":{"key":"dennis@kaarsemaker.net","avatar":"https://avatars.githubusercontent.com/u/200649?v=4"},"body":"On Mon, Jun 02, 2014 at 11:53:36AM -0700, Junio C Hamano wrote:\n> Dennis Kaarsemaker <dennis@kaarsemaker.net> writes:\n> \n> > Instead of calling git-archive HEAD^{tree}, use $(GIT_VERSION)^{tree}.\n> > This makes sure the archive name and contents are consistent, if HEAD\n> > has moved, but GIT-VERSION-FILE hasn't been regenerated yet.\n> >\n> > Signed-off-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>\n> > ---\n> > I have a somewhat odd setup in which I share a .git between multiple\n> > checkouts for automated builds. To minimize locking time for parallel\n> > builds with different options, there's only a lock around checkout and\n> > running git describe $commit > version, the builds themselves run in\n> > parallel.\n> >\n> > This works just fine except during 'make dist', which is hardcoded to\n> > package up HEAD, but that's not always the commit that is actually\n> > checked out, another process may have checked out something else.\n> >\n> > I realize this setup is somewhat strange, but the only change necessary\n> > to make this work is this one-line patch, so I hope that's acceptable.\n> \n> The \"dist\" target happens to do a clean checkout to a temporary\n> directory with \"git archive\" and then muck with its contents before\n> tarring up the result, so moving HEAD around may appear to work for\n> this particular target, but htmldocs/manpages targets use what is in\n> the current checkout of Documentation/ area.  Turning the HEAD^{tree}\n> into $(GIT_VERSION)^{tree} would make the inconsistency between the\n> two worse, wouldn't it?\n\nI'd say it would make the consistency better, because now both look at\nwhat is checked out instead of at HEAD. I agree that it's a game of\nwhack-a-mole though and it's really easy to add another dependency on\nHEAD and GIT-VERSION-FILE agreeing with each other.\n\n> If we were to change the \"dist\" rules, we may want to go in the\n> opposite direction.  Instead of using \"git archive\" to make a\n> temporary copy of a directory from a commit, make such a temporary\n> copy from the contents of the current working tree (or the index).\n> And then tar-up a result after adding configure, version etc. to it.\n> That would mean what will be in the tarball can be different from\n> even HEAD, which is more consistent with the way how documentation\n> tarballs are made.\n> \n> Alternatively, you can update the dist-doc rules to make a temporary\n> copy of documentation area and generate the documentation tarballs\n> out of a pristine source of a known version---which would also make\n> these two consistent.  I am not sure which one is more preferrable,\n> though.\n> \n> Why are you sharing the .git/HEAD across multiple checkouts in the\n> first place?  If they are to build all different versions, surely\n> these working trees are derived from different commits, no?\n\nI'm sharing all of .git using explicit GIT_DIR and GIT_WORK_TREE\nenvironment variables, sharing .git/HEAD comes with that. What I'm\nactually doing is a continuos integration setup that can run many\nactions at once in freshly checked-out work trees, but sharing .git to\nsave space. \n\nWhat it specifically doing is running 'make test' for master, next and\npu, and make dist for next. As long as I protect the 'git checkout\n$sha1' with a lock, that all works just fine.\n\n> Have you considered using contrib/workdir/git-new-workdir, perhaps?\n\nI have not, thanks for the pointer. That approach is definitely cleaner\nthan what I am currently doing.\n\n> I dunno.\n\nWith the hint above, I actually don't need this patch anymore. And if\nyou're not convinced it's a good idea, it's probably better to drop it :)\n\n> >  Makefile | 2 +-\n> >  1 file changed, 1 insertion(+), 1 deletion(-)\n> >\n> > diff --git a/Makefile b/Makefile\n> > index a53f3a8..63d9bac 100644\n> > --- a/Makefile\n> > +++ b/Makefile\n> > @@ -2433,7 +2433,7 @@ git.spec: git.spec.in GIT-VERSION-FILE\n> >  GIT_TARNAME = git-$(GIT_VERSION)\n> >  dist: git.spec git-archive$(X) configure\n> >  \t./git-archive --format=tar \\\n> > -\t\t--prefix=$(GIT_TARNAME)/ HEAD^{tree} > $(GIT_TARNAME).tar\n> > +\t\t--prefix=$(GIT_TARNAME)/ $(GIT_VERSION)^{tree} > $(GIT_TARNAME).tar\n> >  \t@mkdir -p $(GIT_TARNAME)\n> >  \t@cp git.spec configure $(GIT_TARNAME)\n> >  \t@echo $(GIT_VERSION) > $(GIT_TARNAME)/version\n\n-- \nDennis Kaarsemaker <dennis@kaarsemaker.net>\nhttp://twitter.com/seveas\n"},{"id":"243157","messageId":"xmqqd2er3vml.fsf@gitster.dls.corp.google.com","threadId":"36803","inReplyTo":"20140602193419.GA10198@spirit","subject":"Re: [PATCH] Makefile: don't hardcode HEAD in dist target","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2014-06-02T20:27:14Z","receivedAt":"2014-06-02T20:27:14Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Dennis Kaarsemaker <dennis@kaarsemaker.net> writes:\n\n> I'd say it would make the consistency better, because now both look at\n> what is checked out instead of at HEAD.\n\nThe version with your patch does not even look at HEAD; it looks at\nwhatever GIT_VERSION points at, which could be a very different\nversion that does not have anything to do with what is checked out\nCan't GIT_VERSION come from ./version file, for example?\n"}]}