Re: [PATCH] t1410: fix breakage on case-insensitive filesystems

From: Jeff King <peff@peff.net>
Date: Nov 10, 2014, 07:04 UTC
Message-ID: <20141110070459.GC7677@peff.net>
In-Reply-To: <CAPc5daUrUZ135sbSuZn6DXmN_momR6KxNyN9cNdTbzwbD6Uu0A@mail.gmail.com>

On Sun, Nov 09, 2014 at 10:47:52PM -0800, Junio C Hamano wrote:

Show 14 quoted lines

> On Sun, Nov 9, 2014 at 10:30 PM, Jeff King <peff@peff.net> wrote:
> >
> > I know you "make test" before pushing out the results of any integration
> > you do. And I recall that for a while (and maybe still?) you even did so
> > on VMs of a few common platforms. OS X is notoriously irritating to run
> > in a VM, but would you be interested in a network-accessible install
> > that you could push to and "make test" on as part of your routine?
> >
> > If what Michael is offering cannot do that, I am sure I can get GitHub
> > to set something up.
> 
> Even if it were network-accessible, the latency for doing an integration
> cycle and pushing there and waiting for the tests to come back may
> make it impractical to use it as "part of pre-pushout test".

I had the impression you were already doing "for i in my_vms; do git push $i && ssh $i make test; done", so in theory this could be done in parallel. But yeah, I definitely wouldn't want to add latency to your workflow.

Show 5 quoted lines

> But I would think that the project certainly would benefit if a
> post-receive hook at Github did an automated test on the tip of
> 'pu' and bisected a breakage, if found, to pinpoint the patch
> that breaks. It could even send a notice to the author of the
> non-portable patch ;-).

Yeah, you're right. Testing on "pu" is probably enough. Coupled with automated bisection, the "pinpointing" part is not that important, and pu is often "early enough" to catch problems before people try to build on top of them. So I think if Michael is still willing to set up his build box, just pulling and building your "pu" automatically would be a good start.

GitHub supports trigger hooks on pushes, but I do not think that is even necessary here. Just polling once or twice a day would probably give us good enough turn-around time.

I think somebody just needs to write the auto-bisect script. I suspect the simplest form is something like:

  # fetch a new version; exit early if we already checked this one
  git fetch
  test "$(git rev-parse pu-ok)" = "$(git rev-parse origin/pu)" &&exit 0
  git reset --hard origin/pu

  # quick path: all tests succeed
  if make test; then
    git tag -f pu-ok HEAD
    exit 0
  fi

  # slow path: bisect the actual commit. We do not need worry about
  # cutting each bisection step down to the minimal size (e.g., finding
  # the actual failing test), because this should run only infrequently.
  git bisect start HEAD pu-ok
  git bisect run make test

The real work would be in wrapping it up in a cron-job or similar so that it produces no output on success, and sends an email (or whatever) with the bisect results when it fails.

-Peff