Re: Bug? Error during commit

From: Duy Nguyen <pclouds@gmail.com>
Date: Feb 10, 2018, 10:21 UTC
Message-ID: <CACsJy8C72Gv4D46tOmgTqi=8aHVG7D=yZR2F7VPLRcm1e_jTYg@mail.gmail.com>
In-Reply-To: <20180207204520.GA31757@sigill.intra.peff.net>

On Thu, Feb 8, 2018 at 3:45 AM, Jeff King <peff@peff.net> wrote:

Show 41 quoted lines

> On Mon, Feb 05, 2018 at 08:59:52PM +0700, Duy Nguyen wrote:
>
>> On Mon, Feb 5, 2018 at 8:48 PM, Andreas Kalz <andreas-kalz@gmx.de> wrote:
>> > Hello,
>> >
>> > I am using git frequently and usually it is running great.
>> >
>> > I read to write to this eMail address regarding problems and possible bugs.
>> > I am using git version 2.16.1.windows.2 / 64 Bit and during commit the following error message comes up:
>> > e:\Internet>git commit -m 2018-01-27
>> > fatal: unable to generate diffstat for Thunderbird/andreas-kalz.de/Mail/pop.gmx.net/Inbox
>> > [master f74cf30] 2018-01-27
>> >
>> > I also tried this before with an older git version with same problem.
>> >
>> > Can you help me with this problem please? Thanks in advance.
>>
>> I think if you add -q to that "git commit" command, diffstat is not
>> generated and you can get past that. If that particular commit can be
>> published in public, it'll help us find out why diffstat could not be
>> generated.
>
> I think that's the first time I've seen that particular error. :)
>
> I think the only reason that xdiff would report failure is if malloc()
> failed, or if one of the files exceeds MAX_XDIFF_SIZE, which is ~1GB.
> I think we'd usually avoid doing a text diff on anything over
> core.bigFileThreshold, though.
>
> But it doesn't seem to work:
>
>   $ yes | head -c $((1024*1024*1024 - 10*1024*1024)) >file
>   $ git add file
>   $ git commit -m one
>   $ yes | head -c $((1024*1024*1024)) >file
>   $ git commit -am two
>   fatal: unable to generate diffstat for file
>
> What's weird is that if I run "git show --stat" on the same commit, it
> works! So there's something about how commit invokes the diff that
> doesn't let the big-file check kick in.

I have a flashback about checking big_file_threshold in this code. I vaguely remember doing something in this code but not sure if it was merged.

I finally found 6bf3b81348 (diff --stat: mark any file larger than core.bigfilethreshold binary - 2014-08-16) so it's merged after all, but this commit is about --stat apparently ;-)

Show 10 quoted lines

> It looks like the logic in diff_filespec_is_binary() will only check
> big_file_threshold if we haven't already loaded the contents into RAM.
> So "commit" does that, but "diff" is more careful about not loading the
> file contents.
>
> I think we probably ought to consider anything over big_file_threshold
> to be binary, no matter what. Possibly even if the user gave us a
> .gitattribute that says "no really, this is text". Because that 1GB
> limit is a hard limit that the code can't cope with; our options are
> either to generate a binary diff or to die.

Agreed. While at there perhaps we need to improve this "unable to generate diffstat" message a bit too.

-- 
Duy