[POC PATCH 0/5] Threaded loose object and pack access

From: Thomas Rast <trast@student.ethz.ch>
Date: Dec 9, 2011, 08:39 UTC
Message-ID: <cover.1323419666.git.trast@student.ethz.ch>

Well, just to make sure we're all left in a confused mess of partly conflicting patches, here's another angle on the same thing:

Jeff King wrote:

> Wow, that's horrible. Leaving aside the parallelism, it's just terrible
> that reading from the cache is 20 times slower than the worktree. I get
> similar results on my quad-core machine.

By poking around in sha1_file.c I got that down to about 10. It's not great yet, but it seems a start.

The goal would be to improve it to the point where a patch lookup that already has all relevant packs open and windows mapped can proceed without locking. I'm not sure that's doable short of duplicating the whole pack state (including fds and windows) across threads, but I'll give it some more thought before going that route.

Thomas Rast (5):
  Turn grep's use_threads into a global flag
  grep: push locking into read_sha1_*
  sha1_file_name_buf(): sha1_file_name in caller's buffer
  sha1_file: stuff various pack reading variables into a struct
  sha1_file: make the pack machinery thread-safe

 builtin/grep.c   |   60 +++++-----------
 cache.h          |    1 +
 replace_object.c |    5 +-
 sha1_file.c      |  213 +++++++++++++++++++++++++++++++++++++++++-------------
 thread-utils.c   |   30 ++++++++
 thread-utils.h   |   23 ++++++
 6 files changed, 240 insertions(+), 92 deletions(-)

-- 
1.7.8.431.g2abf2