Re: [BUG] index corruption with git commit -p

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: Sep 1, 2018, 22:17 UTC
Message-ID: <87tvn8c166.fsf@evledraar.gmail.com>
In-Reply-To: <20180901214157.hxlqmbz3fds7hsdl@ltop.local>

On Sat, Sep 01 2018, Luc Van Oostenryck wrote:

Show 25 quoted lines

> Hi,
>
> I've just had a scary error:
> 	error: index uses $?+? extension, which we do not understand
> 	fatal: index file corrupt
>
> Things were quickly recovered by deleting the index but it clearly
> looks to a but to me.
>
> Here are the steps to reproduce it:
>   $ git clone git://github.com/lucvoo/sparse-dev.git <somedir>
>   $ cd <somedir>
>   $ git co index-corruption
>   $ git rm -r validation/ Documentation/
>   $ git commit -m <some message> -p
>   $ git status
> error: index uses $?+? extension, which we do not understand
> fatal: index file corrupt
>
>
> The 'extension' pattern '$?+?', can vary a bit, sometimes
> it's just '????', but always seems 4 chars.
> If the commit command doesn't use the '-p' flag, there is no
> problem. The repository itself is not corrupted, it's only
> the index. It happends with git 2.18.0 and 2.17.0

Yeah this is a bug, I didn't dig much but testing with this script down to 2.8.0:

    #!/bin/sh

    cd ~/g/git
    make -j $(parallel --number-of-cores) USE_LIBPCRE2=YesPlease CFLAGS="-O0 -g -ggdb3" DEVELOPER=1 DEVOPTS=no-error NO_OPENSSL=Y all

    (
        rm -rf /tmp/x;
        ~/g/git/git --exec-path=/home/avar/g/git clone git://github.com/lucvoo/sparse-dev.git /tmp/x &&
        cd /tmp/x &&
        ~/g/git/git --exec-path=/home/avar/g/git checkout index-corruption &&
        ~/g/git/git --exec-path=/home/avar/g/git rm -r validation/ Documentation/ &&
        ~/g/git/git --exec-path=/home/avar/g/git commit -p
    )

    ~/g/git/git --exec-path=/home/avar/g/git -C /tmp/x status

    if ~/g/git/git --exec-path=/home/avar/g/git -C /tmp/x status
    then
        exit 0
    else
        exit 1
    fi

I found that the first bad commit was: 680ee550d7 ("commit: skip discarding the index if there is no pre-commit hook", 2017-08-14)

Now, note the two invocations of "status" in my script. Before we'd already been complaining about a bad index, but after that commit is the first time we started getting a persistent error, and indeed even reverting it now on top of master makes the error non-persistent.

So not a fix, but a strong signal to see where we should start looking. I.e. the index file handling around discard_cache() & "interactive" in commit.c is likely what's broken.