[PATCH v3 00/13] Exclude optimizations

Result of today. I cherry-picked nd/read-directory-recursive-optim to see how far I can get after pulling all the tricks. This is a slower machine so time is longer. Anyway, read_directory time is reduced about 70% in the end.

function before after ---------------------------------- treat_leading_path: 0.000 0.000 read_directory: 4.102 1.235 +treat_one_path: 2.843 0.531 ++is_excluded: 2.632 0.102 +++prep_exclude: 0.225 0.040 +++matching: 2.054 0.035 ++dir_exist: 0.035 0.035 ++index_name_exists: 0.292 0.225 lazy_init_name_hash: 0.258 0.155 +simplify_away: 0.085 0.083 +dir_add_name: 0.446 0.000

I don't expect all these patches to go in. The meat is nd/read-directory-recursive-optim (or 10/13) and 09/13. Some other patches are safe even if they do not contribute much to the gain. The last two are probably not worth the trouble.

Nguyễn Thái Ngọc Duy (13):
  dir.c: add MEASURE_EXCLUDE code for tracking exclude performance
  match_pathname: avoid calling strncmp if baselen is 0
  dir.c: inline convenient *_icase helpers
  match_basename: use strncmp instead of strcmp
  match_{base,path}name: replace strncmp_icase with memequal_icase
  dir: pass pathname length to last_exclude_matching
  exclude: avoid calling prep_exclude on entries of the same directory
  exclude: record baselen in the pattern
  exclude: filter out patterns not applicable to the current directory
  read_directory: avoid invoking exclude machinery on tracked files
  Preallocate hash tables when the number of inserts are known in advance
  name-hash: allow to lookup a name with precalculated base hash
  read_directory: calculate name hashes incrementally

 Makefile          |   1 +
 attr.c            |   6 +-
 cache.h           |   2 -
 diffcore-rename.c |   1 +
 dir.c             | 392 ++++++++++++++++++++++++++++++++++++++++++++----------
 dir.h             |  26 +++-
 hash.h            |   7 +
 name-hash.c       |  49 ++++---
 name-hash.h (new) |  45 +++++++
 read-cache.c      |   1 +
 unpack-trees.c    |   1 +
 11 files changed, 431 insertions(+), 100 deletions(-)
 create mode 100644 name-hash.h