Re: [PATCH] t1092: use GIT_PROGRESS_DELAY for consistent results

From: Junio C Hamano <gitster@pobox.com>
Date: May 25, 2021, 02:49 UTC
Message-ID: <xmqq5yz7il39.fsf@gitster.g>
In-Reply-To: <YKwd2e5VxVmU6zqj@nand.local>

Taylor Blau <me@ttaylorr.com> writes:

Show 7 quoted lines

> What would perhaps make more sense is to silence the progress meters
> from the commands themselves. AFAICT the only command called by
> run_on_sparse() which generates a progress meter is 'git checkout',
> 'git merge', and 'git submodule', all of which support '--no-progress'.
>
> Might it be worth passing that option instead of setting
> GIT_PROGRESS_DELAY to a large value?

Yup. The progress meters are obvious source of variation, but there is no guarantee that is the only source of variation. The test strategy to run the same command twice and judge only by observing their stdout and stderr is, eh, suboptimal. If we devise a sequence of commands that are tested immediately followed by a sequence of commands to check the outcome of these commands, and the output from the latter for two runs are the same, then that is a more sensible approach, as the latter "visualize the outcome of the commands that are just tested" can be controlled more tightly to ignore meaningless variations (like progress meters or base-delta paring in a resulting packfile of a parallel packing) and focus on the aspects of the outcome that matter.