Better value for chunk_size when threaded

From: Jon Smirl <jonsmirl@gmail.com>
Date: Dec 6, 2007, 23:58 UTC
Message-ID: <9e4733910712061558k19fbc864ia1fb7a3431fd2603@mail.gmail.com>

I tried some various ideas out for chunk_size and the best strategy I found was to simply set it to a constant. How does 20,000 work on other CPUs?

I'd turn on default threaded support with this change. With threads=1 versus non-threaded there is no appreciable difference in the time.

Is there an API to ask how many CPUs are in the system? It would be nice to default the number of threads equal to the number of CPUs and only use pack.threads=X to override.

Making all of this work by default should help when outside people decide to do a massive import.

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 4f44658..4d73be8 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1645,7 +1645,7 @@ static void ll_find_deltas(struct object_entry
**list, unsigned list_size,
        }

        /* this should be auto-tuned somehow */
-       chunk_size = window * 1000;
+       chunk_size = 20000;

        do {
                unsigned sublist_size = chunk_size;


with chunk_size = 20000, everything is on a q6600 4GB

threads = 5
time git repack -a -d -f --depth=250 --window=250
real    6m20.123s
user    20m25.841s
sys     0m5.520s


threads = 4
time git repack -a -d -f --depth=250 --window=250
real    6m15.525s
user    20m20.852s
sys     0m5.356s

threads = 4
time git repack -a -d -f
real    1m31.537s
user    3m2.063s
sys     0m3.064s

threads = 1
time git repack -a -d -f --depth=250 --window=250
real    18m46.005s
user    18m43.122s
sys     0m1.228s

threads = 1
time git repack -a -d -f
real    2m57.774s
user    2m54.211s
sys     0m1.228s

Non-threaded
time git repack -a -d -f --depth=250 --window=250
real    18m51.183s
user    18m46.538s
sys     0m1.604s

Non-threaded
time git repack -a -d -f
real    2m54.849s
user    2m51.267s
sys     0m1.412s

-- 
Jon Smirl
jonsmirl@gmail.com