Re: [PATCH] update-index: refresh should rewrite index in case of racy timestamps

From: Marc Strapetz <marc.strapetz@syntevo.com>
Date: Dec 23, 2021, 18:24 UTC
Message-ID: <b97672fa-837f-1e28-f7f2-aee80e52d374@syntevo.com>
In-Reply-To: <xmqqfsqkdwo4.fsf@gitster.g>

On 23/12/2021 00:52, Junio C Hamano wrote:

Show 17 quoted lines

> Ah, there are cases where we do clear active_cache_changed when we
> notice that an operation detected an error, to avoid spreading the
> breakage by writing the index file out, and I think that is the
> right thing to do.  Which means that the above patch is not quite
> right.  Perhaps taking all of the above together, something like
> this?
> 
> 	*o->has_errors |= refresh_cache(o->flags | flag);
> 	if (*o->has_errors)
> 		active_cache_changed = 0;
> 	else if (has_racy_timestamps(&the_index))
>          	/*
> 		 * Even if nothing else has changed, updating the file
> 		 * increases the chance that racy timestamps become
> 		 * non-racy, helping future run-time performance.
> 		 */
> 		active_cache_changed |= SOMETHING_CHANGED;

I think it's safe to write the index even if refresh_cache() reports an "error" and we should actually do that:

The underlying refresh_index() will report an "error" only for "file: needs merge" and "file: needs update". In both cases, the corresponding entries will not have been updated. Every entry which has been updated is good on its own and writing these updates makes the index a little bit better. Subsequent calls to refresh_index() won't have to do the same work again (like invoking the quite expensive LFS filter).

This is also how cmd_status() currently works: it does not pay attention to the return value of refresh_index() and will always write the index if racy timestamps are encountered.

Overall, the "error" handling in update-index.c might not always do what one expects. Let's consider your suggested fix. When invoking:

update-index --refresh

this won't fix racy timestamps, however:

update-index --refresh --add untracked

will do. I think this is caused by active_cache_changed being used in two different ways: to indicate that the cache should be written and to indicate that it must not be written. It might be a good idea to take the latter "block index write" to a separate static variable in update-index.c.

Candidate usages of this new "block index write" variable will be in the existing callbacks: errors detected in unresolve_callback() should probably continue to block an index write, to ensure that either all or none of the specified files will be unresolved. For the reupdate_callback(), the underlying do_reupdate() seems to return 0 always, so there is dead code in the callback (or am I completely blind?). To stay on the safe side, we may still continue to block an index write here. The refresh_callback() will never block an index write.

Does it make sense to clarify error handling in some preceding commit and only then address the razy timestamps? It will probably make this second commit clearer.

Show 14 quoted lines

>> +}
>> +
>> +update_assert_changed() {
>> +	local ts1=$(test-tool chmtime --get .git/index) &&
>> +	test_might_fail git update-index $1 &&
>> +	local ts2=$(test-tool chmtime --get .git/index) &&
>> +	[ $ts1 -ne $ts2 ]
>> +}
>> +
>> +test_expect_success 'setup' '
>> +	touch .git/fs-tstamp &&
> 
> Not that it is wrong, but do we need to create such a throw-away
> file inside the .git directory?

We actually only need a timestamp for which we know that it is before the timestamp the next file system operation would create. I agree that it should be easy to rewrite that using "test-tool chmtime". This should also simplify reset_mtime().

Regarding all other comments, thanks, I'll address them as suggested for the next patch. And sorry for not checking CodingGuidelines before (I had completely missed this document).

-Marc