[PATCH] Prevent git from rehashing 4GBi files
- From
- Jason D. Hatton <jason.hatton@gmail.com>
- Date
- May 7, 2022, 18:58 UTC
- Message-ID
- <20220507185813.1403802-1-jhatton@globalfinishing.com>
- In-Reply-To
- <philipoakley@iee.email>
Git cache stores file sizes using uint32_t. This causes any file that is a multiple of 2^32 to have a cached file size of zero. Zero is a special value used by racily clean. This causes git to rehash every file that is a multiple of 2^32 every time git status or git commit is run.
This patch mitigates the problem by making all files that are a multiple of 2^32 appear to have a size of 1 instead of zero.
Signed-off-by: Jason D. Hatton <jhatton@globalfinishing.com> --- read-cache.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-)
diff --git a/read-cache.c b/read-cache.c index 4df97e185e..d80c80ef90 100644 --- a/read-cache.c +++ b/read-cache.c @@ -163,6 +163,22 @@ void rename_index_entry_at(struct index_state *istate, int nr, const char *new_n add_index_entry(istate, new_entry, ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE); } +/* + * stat_data only stores file sizes as an unsigned int. Any file that is an + * exact multiple of 4GiB will get a cached file size of zero. Unfortunately, + * this is a special flag used by the racy update logic. Substitute a new file + * size if a non-zero sized file would would be cached as zero. 1U<<31 is used + * as the substitute because it is the furthest away from 0 and 4GiB. + */ +static inline unsigned int munge_st_size(off_t st_size) { + unsigned int sd_size = (unsigned int) st_size; + + if (!sd_size && st_size) + return 1U<<31; + else + return sd_size; +} + void fill_stat_data(struct stat_data *sd, struct stat *st) { sd->sd_ctime.sec = (unsigned int)st->st_ctime; @@ -173,7 +189,7 @@ void fill_stat_data(struct stat_data *sd, struct stat *st) sd->sd_ino = st->st_ino; sd->sd_uid = st->st_uid; sd->sd_gid = st->st_gid; - sd->sd_size = st->st_size; + sd->sd_size = munge_st_size(st->st_size); } int match_stat_data(const struct stat_data *sd, struct stat *st) @@ -212,7 +228,7 @@ int match_stat_data(const struct stat_data *sd, struct stat *st) changed |= INODE_CHANGED; #endif - if (sd->sd_size != (unsigned int) st->st_size) + if (sd->sd_size != munge_st_size(st->st_size)) changed |= DATA_CHANGED; return changed;
-- 2.36.0.3