Re: [BUG] index corruption with git commit -p

From: Jeff King <peff@peff.net>
Date: Sep 2, 2018, 05:08 UTC
Message-ID: <20180902050803.GA21324@sigill.intra.peff.net>
In-Reply-To: <87tvn8c166.fsf@evledraar.gmail.com>

On Sun, Sep 02, 2018 at 12:17:53AM +0200, Ævar Arnfjörð Bjarmason wrote:

Show 21 quoted lines

> > Here are the steps to reproduce it:
> >   $ git clone git://github.com/lucvoo/sparse-dev.git <somedir>
> >   $ cd <somedir>
> >   $ git co index-corruption
> >   $ git rm -r validation/ Documentation/
> >   $ git commit -m <some message> -p
> >   $ git status
> > error: index uses $?+? extension, which we do not understand
> > fatal: index file corrupt
> >
> > The 'extension' pattern '$?+?', can vary a bit, sometimes
> > it's just '????', but always seems 4 chars.
> > If the commit command doesn't use the '-p' flag, there is no
> > problem. The repository itself is not corrupted, it's only
> > the index. It happends with git 2.18.0 and 2.17.0
> 
> Yeah this is a bug, I didn't dig much but testing with this script down
> to 2.8.0:
> [...]
> I found that the first bad commit was: 680ee550d7 ("commit: skip
> discarding the index if there is no pre-commit hook", 2017-08-14)

I think it's much older than that. I set up my test repo like this:

  git clone git://github.com/lucvoo/sparse-dev.git
  cd sparse-dev
  git checkout --detach

and then bisected with this script:

  cd /path/to/sparse-dev
  rm .git/index
  git reset --hard index-corruption &&
  git rm -q -r validation/ Documentation/ &&
  git commit -qm foo -p &&
  git status

Since a33fc72fe9 (read-cache: force_verify_index_checksum, 2017-04-14), that produces the corrupt extension error. But before that, I consistently get:

  error: bad index file sha1 signature
  fatal: index file corrupt

from git-commit. And that bisects back to 9c4d6c0297 (cache-tree: Write updated cache-tree after commit, 2014-07-13).

If I revert that commit (which takes some untangling, see below), then the problem seems to go away. Here's the patch I tried on top of the current master, though I think it is actually the first hunk that is making the difference.

---

diff --git a/builtin/commit.c b/builtin/commit.c
index 0d9828e29e..779c5e2cb5 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -359,13 +359,6 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix
 
 		discard_cache();
 		read_cache_from(get_lock_file_path(&index_lock));
-		if (update_main_cache_tree(WRITE_TREE_SILENT) == 0) {
-			if (reopen_lock_file(&index_lock) < 0)
-				die(_("unable to write index file"));
-			if (write_locked_index(&the_index, &index_lock, 0))
-				die(_("unable to update temporary index"));
-		} else
-			warning(_("Failed to update main cache tree"));
 
 		commit_style = COMMIT_NORMAL;
 		ret = get_lock_file_path(&index_lock);
@@ -408,9 +401,6 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix
 	if (!only && !pathspec.nr) {
 		hold_locked_index(&index_lock, LOCK_DIE_ON_ERROR);
 		refresh_cache_or_die(refresh_flags);
-		if (active_cache_changed
-		    || !cache_tree_fully_valid(active_cache_tree))
-			update_main_cache_tree(WRITE_TREE_SILENT);
 		if (write_locked_index(&the_index, &index_lock,
 				       COMMIT_LOCK | SKIP_IF_UNCHANGED))
 			die(_("unable to write new_index file"));
@@ -457,7 +447,6 @@ static const char *prepare_index(int argc, const char **argv, const char *prefix
 	hold_locked_index(&index_lock, LOCK_DIE_ON_ERROR);
 	add_remove_files(&partial);
 	refresh_cache(REFRESH_QUIET);
-	update_main_cache_tree(WRITE_TREE_SILENT);
 	if (write_locked_index(&the_index, &index_lock, 0))
 		die(_("unable to write new_index file"));
 

-Peff