[RFC 0/16] Introduce index file format version 5

Series of patches to introduce the index version 5 file format. This series does not include any fancy stuff like partial loading or partial writing yet, though it's possible to do that with the new format.

There was already a POC for partial loading, which gave pretty good results, which was however broken in all but the general case, so it's not included yet. (for timings see: http://thread.gmane.org/gmane.comp.version-control.git/201964/focus=202019)

The first 4 patches are refactoring the old code, splitting it up into different functions, as a preparation for index-v5.

Patches 5 and 6 fix testcases for index v5.

Patch 9..11 introduce the reader for index-v5. I've split those patches up to read the main index first, then the resolve-undo data and then the cache-tree, to make it easier to review them.

The same goes for patches 12..14, which introduce the writer, again split up in writing the main index, resolve-undo data and cache-tree data.

Patch 15 adds a option to update index to force-rewrite the index, so rewriting it even if nothing has changed. This is later used for performance testing, to test the performance for both the reader and the writer.

Patch 16 adds the performance test, which compares the time for force-rewrites for index-v[23], index-v4 and index-v5.

The default index format is still set to 3, it can be changed in read-cache.c (INDEX_FORMAT_DEFAULT)

[PATCH 01/16] Modify cache_header to prepare for other index formats [PATCH 02/16] Modify read functions to prepare for other index [PATCH 03/16] Modify match_stat_basic to prepare for other index [PATCH 04/16] Modify write functions to prepare for other index [PATCH 05/16] t2104: Don't fail when index version is 5 [PATCH 06/16] t3700: sleep for 1 second, to avoid interfering with [PATCH 07/16] Add documentation of the index-v5 file format [PATCH 08/16] Make in-memory format aware of stat_crc [PATCH 09/16] Read index-v5 [PATCH 10/16] Read resolve-undo data [PATCH 11/16] Read cache-tree in index-v5 [PATCH 12/16] Write index-v5 [PATCH 13/16] Write index-v5 cache-tree data [PATCH 14/16] Write resolve-undo data for index-v5 [PATCH 15/16] update-index.c: add a force-rewrite option [PATCH 16/16] p0002-index.sh: add perf test for the index formats

Documentation/technical/index-file-format-v5.txt | 281 ++++++++++++++++++++++++++++++++++ builtin/update-index.c | 5 +- cache-tree.c | 145 ++++++++++++++++++ cache-tree.h | 7 + cache.h | 96 +++++++++++- read-cache.c | 1519 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------- resolve-undo.c | 129 ++++++++++++++++ resolve-undo.h | 3 + t/perf/p0002-index.sh | 33 ++++ t/t2104-update-index-skip-worktree.sh | 15 +- t/t3700-add.sh | 1 + test-index-version.c | 2 +- 12 files changed, 2082 insertions(+), 154 deletions(-)