patchp4000: use -3000 when promising -3000

The 'log -3000 (baseline)' test accidentally still used -1000 from an earlier version.

Noticed-by: Lawrence Holding <Lawrence.Holding@cubic.com>
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
---

That has to be the worst paperbag bug in quite some time, especially considering that this is already in master and -rc0. Luckily it doesn't affect the validity of the *actual* tests, and thus what the commit message said.

Going off on a tangent: I played with bisecting for performance regressions yesterday, and this shows a bit of a problem with all of these tests.

I could use an unrestricted 'log' here, but then the perf suite would be too slow to "just run" with defaults (i.e., on git.git). For example, if I run p4000 with 'sed -i s/-3000//' applied I get

  Test                             this tree        
  --------------------------------------------------
  4000.1: log  (baseline)          0.59(0.52+0.06)  
  4000.2: log --raw  (tree-only)   3.02(2.57+0.43)  
  4000.3: log -p  (Myers)          17.66(16.65+0.93)
  4000.4: log -p  --histogram      17.64(16.78+0.79)
  4000.5: log -p  --patience       20.71(19.68+0.96)

For best-of-3 runs (the default) that's still a runtime on the order of 3min -- for a single test!

The -3000 is also plenty, on my box, to prove (statistically, with significance to spare) an improvement with this hack for the diff engine:

  https://github.com/trast/git/commit/a0acba9b853ae257260fd7bd2f1a81389e1c75e8

OTOH I had some problems with completely unattended bisections. Consider the results, for the fixed version:

  Test                                  this tree      
  -----------------------------------------------------
  4000.1: log -3000 (baseline)          0.08(0.07+0.01)
  4000.2: log --raw -3000 (tree-only)   0.38(0.33+0.04)
  4000.3: log -p -3000 (Myers)          1.70(1.58+0.11)
  4000.4: log -p -3000 --histogram      1.67(1.57+0.09)
  4000.5: log -p -3000 --patience       2.02(1.89+0.12)

Suppose I was bisecting for a 5% performance hit. On the 1.70s timings, that would be about 90ms, but the noise in the timings can easily be 30-50ms. So there is a gap between "nice for humans to just run" and "great S/N". "Reasonable S/N" won't do it for a bisection, as a single bad decision sends you the wrong way.

I did, however, get good results by removing the -3000 limits and then automating a bisection for an 8% performance hit in 4000.3 between v1.7.0 and master, which turns out to be explained by 3e5a188 (diff.c: Ensure "index $from..$to" line contains unambiguous SHA1s, 2010-05-30).

So perhaps there should be a knob that says "and now take 10 times longer please". Or something like that.

 t/perf/p4000-diff-algorithms.sh |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)