git bisect on multiple cores

git bisect is really cool. I use it for the first time. Just a thought, if you have a multiple core computer, can't you make git build new versions in the background while testing the previuos version? Alright, if you build 2 versions, one of them will never be tested, but you will perhaps save some time by letting it build in the background?

Just a thought...

On Tue, Apr 8, 2008 at 8:58 PM, A B <gentosaker@gmail.com> wrote:

Git is used to track more than just source code that can be "built", and git bisect can be used for more than just tracking down bugs.

I'm not convinced the considerable extra complexity would be worthwhile. You'd have to have git bisect do checkouts to new temporary directories, track them and clean them up.

It might be interesting if you approached it as a tri-section or a general N-section where you try to divide the interval into N parts and concurrently test N-1 commits. But really, do you find git bisect all that slow in practice? You probably have a reasonable guess as to where a regression has come in, and so even 1000 revisions needs at most 10 bisections to find the culprit.

Dave.

"David Symonds" <dsymonds@gmail.com> writes:

I personally think "git bisect" Porcelain does a bit too much. For example, it always checks the revision out, but some non-build checks can be done without having a checkout.

The core idea of bisect, which is really about how to effectively use "git rev-list --bisect", is quite simple. You start by one bad and zero or more "good" ones, where the "bad" one is a descendant of the good ones, and hunt for a _single_ change that changed a "good" state into a "bad" one.

This last point is important. The only thing "bisect" can find is a single flip where all of its ancestors are "good" and where all of its descendants are "bad". Think of a bad gene introduced by a mutation at a particular commit (i.e. "generation") and that contaminates all of its children and descendants forever.

If you visualize the commit ancestry graph growing from left to right like we usually draw in our documentation and e-mails, and if you paint known-to-be-good ones blue, known-to-be-bad ones red, and unknown ones yellow, you will get a picture not unlike the ones shown on pp.111-115 of http://userweb.kernel.org/~junio/200607-ols.pdf

The underlying "git rev-list --bisect" takes a set of "good" commits and a single "bad" commit, computes the set of commits that are unknown ("yellow"), and gives one of them that lies halfway from "good" ones ("blue") and the "bad" one ("red").

The way "git bisect" operates is:

 (0) Prime the process by getting a single "bad" and zero or more "good";
     switch to "bisect" branch.

 (1) Ask "rev-list --bisect" the midpoint; check out that revision for you
     to test.

 (2) It's your turn to give more information to "git bisect".

 (2-a) If you say "good", it is added to the set of "good"; go back to (1)

 (2-b) If you say "bad", it is set to "bad" (because of the way bisection
       works, this is always an ancestor of the previous "bad", and
       because the only thing we do is to find a single flip, keeping a
       single "bad" that is an ancestor of all the commits previously
       known-to-be-bad is sufficient); go back to (1)

One thing to note is that in (2-a) or (2-b), you do not necessarily have to say the commit the command gave you in step (1) is good or bad. If the revision given by (1) is untestable, you can reset to another yellow one, test that, and tell the command "This is good/bad".

So one way to speed up your bisection process would be:

 * Have multiple work trees (e.g. contrib/workdir/git-new-workdir);

 * Run bisect in one repository;

 * In step (1) of each round, look at gitk output and pick another commit
   that is distant from the one you are going to test.  In another work
   tree, check that one out and test it in parallel.

 * You can feed the good/bad information you obtained from the test you
   run in the neighbouring work tree, in addition to what you learn in
   your main tree, with "git bisect good $it" or "git bisect bad $it".

I think adding N-section to "rev-list --bisect" would generally be an interesting thing to do. For one thing, it would allow you to automate the step to "pick another commit that is distant from the one you are going to test" in the above sequence.

Le mardi 8 avril 2008, A B a écrit :

Yes, you can do that.

If you have cloned your repository twice (or more), then you can bisect compile and test at the same time in your 3 repositories (or more if you really want).

For example if bisecting in one repo asks you to test revision X, then you can bisect and then build (and even maybe start testing) in another repo assuming revision X is good, and in yet in another one assuming revision X is bad.

(In the repo where you assume X is good you use:

"git bisect start CURRENT-BAD X"

and in the third one, where you assume X is bad, you use:

"git bisect start X CURRENT-GOOD")

When you know that X was good then you can kill the build or test processes and "git bisect reset" in the repository where you assumed wrongly X was bad. You can then assume something else with "git bisect start B G" and build and test in this repo again.

In the repo that told you X was good, then you need only to use "git bisect good" or "git bisect bad" without building and testing to assume something about the revision that should be built and tested.

Regards, Christian.

Le mercredi 9 avril 2008, Christian Couder a écrit :

In fact, you have to give all the good revisions you have, not just the last one. So it should be

"git bisect start CURRENT-BAD X GOOD1 GOOD2 ..."

or

"git bisect start X GOOD1 GOOD2 ..."

instead of the above.

Above also you must give all the good ones, so it is:

"git bisect start B G1 G2..."

Regards, Christian.

On 2008-04-08 12:58:47 +0200, A B wrote:

Adding parallelism to a binary search scales very badly -- I'd say about logarithmically, but I haven't thought hard about it. If it's possible to use the extra cores to speed up the build+test cycle, that's vastly preferable.

Karl Hasselström <kha@treskal.com> writes:

Probably logarithmically with the number of cores. But for reasonable machines, this number is relatively low, so the log is not so costly. For a binary search, using just 2 cores, you can try the next in the list in case of a "git bisect good" for example, and if the hypothesis is true, you've just gained a factor 2 (assuming it happens 50% of times, that should be a 50% speedup). Similarly, you should get a factor 2 with 3 cores.

And as said before, you can also try trisection or N-section. But no, I won't claim this is simple to implement ;-).

On 2008-04-09 10:07:25 +0200, Matthieu Moy wrote:

Yeah. But to get a factor 3, you need 7 cores; and for 4, you need 15. It goes downhill from there. If your build+test cycle is parallelizable at all, I don't think you'll find those numbers hard to beat.

(There's also the fact that testing several revisions at once assumes that the whole build+test cycle is automated, or at least most of it. Otherwise you need more people as well as more cores.)