Re: multiple working directories for long-running builds

From: Junio C Hamano <gitster@pobox.com>
Date: Dec 2, 2009, 02:09 UTC
Message-ID: <7viqcqp1nh.fsf@alter.siamese.dyndns.org>
In-Reply-To: <20091201211830.GE11235@dpotapov.dyndns.org>

Dmitry Potapov <dpotapov@gmail.com> writes:

Show 18 quoted lines

> On Tue, Dec 01, 2009 at 01:58:05PM -0500, Greg A. Woods wrote:
>> 
>> > > I just disagreed that "git archive" was a reasonable alternative to
>> > > leaving the working directory alone during the entire time of the build.
>> > 
>> > Using "git archive" allows you avoid running long time procedure such as
>> > full clean build and testing in the working tree. Also, it is guaranteed
>> > that you test exactly what you put in Git and some other garbage in your
>> > working tree does not affect the result.
>> 
>> Sure, but let's be very clear here:  "git archive" is likely even more
>> impossible for some large projects to use than "git clone" would be to
>> use to create build directories.
>
> AFAIK, "git archive" is cheaper than git clone. I do not say it is fast
> for huge project, but if you want to run a process such as clean build
> and test that takes a long time anyway, it does not add much to the
> total time.

I do not understand people who advocate for "git archive" to be used in this manner at all.

I do use a set of separate build directories, and I typically run 5 to 10 full builds (in each) per day, but I rarely if ever make fix in them. Perhaps the usage pattern expected by people who want others to use "git archive" to prepare separate build directories may be different from how I use them for.

I see two downsides in using "git archive":

 - "archive" piped to "tar xf -" will overwrite _all_ files every time you
   refresh the build area, causing extra work on "make" and any build
   procedure based on file timestamps.  Sure, you can work it around by
   using ccache but why make your life complicated?

 - When a build in these separate build areas fails, you would want to go
   there and try to diagnose or even fix the problem in there, not in your
   primary working area (after all, the whole point of keeping a separate
   build area is so that you do not have to switch branches too much in
   the primary working area).  A directory structure prepared by "archive"
   piped to "tar xf -" however is not a work tree, and any experimental
   changes (e.g. "debugf()") or fixes you make there need to be reverted
   or taken back manually to be placed in the primary working area.

If your build area is prepared with new-workdir, then you share the history and you even share the ref namespace, so that "reset --hard" will remove all the debugf() added while diagnosing, and "diff" will give you the patch you need to take home.

You could even make a commit from your build area, but this cuts both ways. You need to be aware that after committing on a branch in one repository other repositories that have the same branch checked out will become out of sync. It is however less of an issue in practice, because the build areas are typically used to check out integration branches (e.g. 'master' and 'next' in git.git) that you do not directly commit anyway, and you will get very aware of the tentative nature of the tree, as the update procedure for such a build area prepared with new-workdir is always:

    cd /buildfarm/<branch>/ && git reset --hard

This will not touch any file that do not have to get updated, so your "make" won't get confused.