[PATCH 0/5] multithread traverse_commit_list (aka rev-list)

rev-list is quite cpu extensive. While there's better approach to tackle this (e.g. Shawn's bitmap cache idea), I was curious if making it parellel could give us anything. So this is a naive (and failed) attempt at that. I post it anyway, so people, especially gsoc students if the related proposal is chosen, might learn from it, or learn to avoid it, or just laugh at my attempt to relieve stress.

While this series does not break rev-list (but breaks all external commands, I don't care anyway), and it proves lock-free pack access works, it does not improve rev-list performance. I suspect extensive locking around "struct object *" may be the culprit. Or maybe it's just my dumb partition method. A better partition method that avoids locking might improve it. We may split the object hash table into 4, one for each object type and assign one thread for tree, one for blob. Or something else..

Anyway I'll stop here to finish my ongoing topics for 1.7.11 and maybe implement Shawn's idea if I still have more time. Effectively generating the cache from index-pack could be challenging.

Nguyễn Thái Ngọc Duy (5):
  Remove global pointer "packed_git" in favor or set/get function pair
  sha1_file: stuff various pack reading variables into a struct
  Make lookup_*() functions thread-safe
  Teach traverse_commit_list callsites about new parameter, nr_threads
  Support multithread in traverse_commit_list and rev-list

 blob.c                   |   11 ++-
 builtin/count-objects.c  |    4 +-
 builtin/fsck.c           |    4 +-
 builtin/gc.c             |    2 +-
 builtin/pack-objects.c   |   12 +-
 builtin/pack-redundant.c |    4 +-
 builtin/rev-list.c       |    5 +-
 cache.h                  |    8 +-
 commit.c                 |   12 ++-
 git.c                    |    1 +
 list-objects.c           |  157 +++++++++++++++++++++++++-
 list-objects.h           |    2 +-
 object.c                 |    9 ++-
 object.h                 |    6 +
 pack-revindex.c          |    4 +-
 revision.c               |   16 +++
 revision.h               |    2 +
 server-info.c            |    4 +-
 sha1_file.c              |  276 ++++++++++++++++++++++++++++------------------
 sha1_name.c              |    2 +-
 tag.c                    |    9 +-
 tree.c                   |   18 +++-
 upload-pack.c            |    2 +-
 23 files changed, 419 insertions(+), 151 deletions(-)