Re: Git status extremely slow if any file is a multiple of 8GBi

From: Junio C Hamano <gitster@pobox.com>
Date: May 4, 2022, 13:55 UTC
Message-ID: <xmqqczgtpfl5.fsf@gitster.g>
In-Reply-To: <CY4PR16MB1655A1F7DD8B2FABCC30C9DAAFC39@CY4PR16MB1655.namprd16.prod.outlook.com>

Jason Hatton <jhatton@globalfinishing.com> writes:

Show 5 quoted lines

> I have a proposed idea that may or may not help. Would it be possible for any
> file that is a multiple of 2^32 to be adjusted to a file size of 1 instead of
> zero? Git is already functioning with mangling file sizes over 4GB in the
> index, so maybe bumping up the size of 2^32 multiple files would mitigate the
> issue.

Clever.

The condition sd_size==0 is used as a signal for "no, we really need to compare the contents", and causes the contents to be hashed, and if the contents match the object name recorded in the index, the on-disk size is stored in sd_size and the entry is marked as CE_UPTODATE. Alas, if the truncated st_size is 0, the resulting entry would have sd_size==0 again, so a workaround like what you outlined is needed.

You'd need to make sure that you tweaked

	if (sd->sd_size != (unsigned int) st->st_size)
		changed |= DATA_CHANGED;

that appears at the end of read-cache.c::match_stat_data() in a way that is consistent with how you munge the sd_size member in fill_stat_data(), if you are going to go that route.

Thanks.