Re: [PATCH 2/2] read-cache: fix incorrect count and progress bar stalling

From: René Scharfe <l.s.r@web.de>
Date: Jun 8, 2021, 16:14 UTC
Message-ID: <74183ce6-e17f-1b11-1ceb-7a8d873bc1c7@web.de>
In-Reply-To: <87wnr4394y.fsf@evledraar.gmail.com>

Am 08.06.21 um 12:58 schrieb Ævar Arnfjörð Bjarmason:

Show 43 quoted lines

>
> On Tue, Jun 08 2021, Junio C Hamano wrote:
>
>> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:
>>
>>>> So I think this pattern works:
>>>>
>>>> 	for (i = 0; i < nr; i++) {
>>>> 		display_progress(p, i);
>>>> 		/* work work work */
>>>> 	}
>>>> 	display_progress(p, nr);
>>>>
>>>> Alternatively, if the work part doesn't contain continue statements:
>>>>
>>>> 	for (i = 0; i < nr; i++) {
>>>> 		/* work work work */
>>>> 		display_progress(p, i + 1);
>>>> 	}
>>>
>>> But yes, I agree with the issue in theory, but I think in practice we
>>> don't need to worry about these 100% cases.
>>
>> Hmph, but in practice we do need to worry, don't we?  Otherwise you
>> wouldn't have started this thread and René wouldn't have responded.
>
> I started this thread because of:
>
> 	for (i = 0; i < large_number; i++) {
> 		if (maybe_branch_here())
> 			continue;
> 		/* work work work */
> 		display_progress(p, i);
> 	}
> 	display_progress(p, large_number);
>
> Mainly because it's a special snowflake in how the process.c API is
> used, with most other callsites doing:
>
> 	for (i = 0; i < large_number; i++) {
> 		display_progress(p, i + 1);
> 		/* work work work */
> 	}

Moving the first call to the top of the loop makes sense. It ensures all kind of progress -- skipping and actual work -- is reported without undue delay.

Adding one would introduce an off-by-one error. Removing the call after the loop would leave the progress report at one short of 100%. I don't see any benefits of these additional changes, only downsides.

If other callsites have an off-by-one error and we care enough then we should fix them. Copying their style and spreading the error doesn't make sense -- correctness trumps consistency.

Show 5 quoted lines

> Fair enough, but in the meantime can we take this patch? I think fixing
> that (IMO in practice hypothetical issue) is much easier when we
> consistently use that "i + 1" pattern above (which we mostly do
> already). We can just search-replace "++i" to "i++" and "i + 1" to "i"
> and have stop_progress() be what bumps it to 100%.

This assumes the off-by-one error is consistent. Even if that is the case you could apply your mechanical fix and leave out read-cache. This would happen automatically because when keeping i there is no ++i to be found.

stop_progress() doesn't set the progress to 100%:

   $ (echo progress 0; echo update) |
     ./t/helper/test-tool progress --total 1 test
   test:   0% (0/1), done.

I wonder (only in a semi-curious way, though) if we can detect off-by-one errors by adding an assertion to display_progress() that requires the first update to have the value 0, and in stop_progress() one that requires the previous display_progress() call to have a value equal to the total number of work items. Not sure it'd be worth the hassle..

René