[PATCH/WIP 0/7] was: long fast-import errors out "failed to apply delta"

A very short summary. It was found [1] that fast-import sometimes can produce broken packfiles (sha1 mismatch) or even wrong packfiles (data differs from what was expected). This happens mostly on not so tiny svn-to-git or even cvs-to-svn-to-git imports with all these copying across the tree (simulating tags/branches as a directories in git, for example). But I won't be surprised if this can happen without these operations too.

Technically, fast-import has in-memory tree representation where it stores sha1's of some previous tree states (to make delta on them), but when it comes to producing the delta, old sha1's tree content is fetched from the in-memory node and it's children (not via sha1->object lookup). And these can turn out to be unrelated to each other as some operations changes the children's states.

The most wanted bit for these patches is small testcases. Keeping in mind all the in-memory tree state and fast-import logic is hard for me, so I wasn't able to create small tests (the best is [2] - 15M archive + custom git builds + fix the Makefile in [2] + a few minutes to reproduce). Another good todo is to always avoid base sha1's mismatch (not just to avoid corruption if it is detected). I think I can do this, but I won't be sure in the code unless there is a bunch of good tests, this series is quite big already.

[1] http://thread.gmane.org/gmane.comp.version-control.git/176753 [2] http://thread.gmane.org/gmane.comp.version-control.git/176753/focus=177901

Dmitry Ivankov (7):
  fast-import: extract object preparation function
  fast-import: be saner with temporary trees
  fast-import: fix a data corruption in parse_ls
  fast-import: fix data corruption in store_tree
  fast-import: extract tree_content reading function
  fast-import: workaround data corruption
  fast-import: fix data corruption in load_tree

 fast-import.c |  169 +++++++++++++++++++++++++++++++++++++++++++++------------
 1 files changed, 135 insertions(+), 34 deletions(-)