git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 7/7] ci: run test round with parallel-checkout enabled

From
Derrick Stolee <stolee@gmail.com>
Date
Apr 23, 2021, 19:56 UTC
Message-ID
<56757f2b-f203-3014-dbad-e86c49d0d64b@gmail.com>
In-Reply-To
<f7c20e0b5bd7857b2303aab762f24c503e50030d.1619104091.git.matheus.bernardino@usp.br>
On 4/22/2021 11:17 AM, Matheus Tavares wrote:
Show 19 quoted lines
> We already have tests for the basic parallel-checkout operations. But
> this code can also run be executed by other commands, such as
> git-read-tree and git-sparse-checkout, which are currently not tested
> with multiple workers. To promote a wider test coverage without
> duplicating tests:
> 
> 1. Add the GIT_TEST_CHECKOUT_WORKERS environment variable, to optionally
>    force parallel-checkout execution during the whole test suite.
> 
> 2. Set this variable (with a value of 2) in the second test round of our
>    linux-gcc CI job. This round runs `make test` again with some
>    optional GIT_TEST_* variables enabled, so there is no additional
>    overhead in exercising the parallel-checkout code here.
> 
> Note that tests checking out less than two parallel-eligible entries
> will fall back to the sequential mode. Nevertheless, it's still a good
> exercise for the parallel-checkout framework as the fallback codepath
> also writes the queued entries using the parallel-checkout functions
> (only without spawning any worker).
I agree that this is helpful additional testing.
Show 19 quoted lines
> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
> ---
>  ci/run-build-and-tests.sh  |  1 +
>  parallel-checkout.c        | 14 ++++++++++++++
>  t/README                   |  4 ++++
>  t/lib-parallel-checkout.sh |  3 +++
>  4 files changed, 22 insertions(+)
> 
> diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
> index a66b5e8c75..23b28e7391 100755
> --- a/ci/run-build-and-tests.sh
> +++ b/ci/run-build-and-tests.sh
> @@ -25,6 +25,7 @@ linux-gcc)
>  	export GIT_TEST_ADD_I_USE_BUILTIN=1
>  	export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=master
>  	export GIT_TEST_WRITE_REV_INDEX=1
> +	export GIT_TEST_CHECKOUT_WORKERS=2
>  	make test
>  	;;

(Making a note to audit the entries here, as I believe some of these could be enabled by default or no longer do anything.)

Show 6 quoted lines
>  void get_parallel_checkout_configs(int *num_workers, int *threshold)
>  {
> +	char *env_workers = getenv("GIT_TEST_CHECKOUT_WORKERS");
> +
> +	if (env_workers && *env_workers) {
> +		if (strtol_i(env_workers, 10, num_workers)) {

I had to look up how strtol_i() differs from strtol(). It seems like we should use strtol_i() more often than we do.

Show 9 quoted lines
> +			die("invalid value for GIT_TEST_CHECKOUT_WORKERS: '%s'",
> +			    env_workers);
> +		}
> +		if (*num_workers < 1)
> +			*num_workers = online_cpus();
> +
> +		*threshold = 0;
> +		return;
> +	}
LGTM.

Thanks, -Stolee

Previous: Matheus TavaresNext: Matheus Tavares
Message 16 of 50 in “Parallel Checkout (part 3)”
  1. 0/7 Parallel Checkout (part 3)Matheus Tavares, Apr 22, 2021
  2. 1/7 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Apr 22, 2021
  3. 2/7 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Apr 22, 2021
  4. Derrick StoleeApr 23, 2021
  5. Matheus Tavares BernardinoApr 26, 2021
  6. 3/7 checkout-index: add parallel checkout supportMatheus Tavares, Apr 22, 2021
  7. Derrick StoleeApr 23, 2021
  8. Matheus Tavares BernardinoApr 26, 2021
  9. 4/7 parallel-checkout: add tests for basic operationsMatheus Tavares, Apr 22, 2021
  10. Derrick StoleeApr 23, 2021
  11. Matheus Tavares BernardinoApr 27, 2021
  12. 5/7 parallel-checkout: add tests related to path collisionsMatheus Tavares, Apr 22, 2021
  13. 6/7 parallel-checkout: add tests related to .gitattributesMatheus Tavares, Apr 22, 2021
  14. Derrick StoleeApr 23, 2021
  15. 7/7 ci: run test round with parallel-checkout enabledMatheus Tavares, Apr 22, 2021
  16. Derrick StoleeApr 23, 2021
  17. 0/8 Parallel Checkout (part 3)Matheus Tavares, Apr 30, 2021
  18. 1/8 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, Apr 30, 2021
  19. Christian CouderMay 1, 2021
  20. Matheus Tavares BernardinoMay 3, 2021
  21. 2/8 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, Apr 30, 2021
  22. Christian CouderMay 1, 2021
  23. Matheus Tavares BernardinoMay 3, 2021
  24. 3/8 checkout-index: add parallel checkout supportMatheus Tavares, Apr 30, 2021
  25. Christian CouderMay 1, 2021
  26. Matheus Tavares BernardinoMay 3, 2021
  27. 4/8 parallel-checkout: add tests for basic operationsMatheus Tavares, Apr 30, 2021
  28. 5/8 parallel-checkout: add tests related to path collisionsMatheus Tavares, Apr 30, 2021
  29. Torsten BögershausenMay 2, 2021
  30. Matheus Tavares BernardinoMay 3, 2021
  31. 6/8 t0028: extract encoding helpers to lib-encoding.shMatheus Tavares, Apr 30, 2021
  32. 7/8 parallel-checkout: add tests related to .gitattributesMatheus Tavares, Apr 30, 2021
  33. 8/8 ci: run test round with parallel-checkout enabledMatheus Tavares, Apr 30, 2021
  34. Torsten BögershausenMay 2, 2021
  35. Matheus Tavares BernardinoMay 3, 2021
  36. 0/8 Parallel Checkout (part 3)Matheus Tavares, May 4, 2021
  37. 1/8 make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares, May 4, 2021
  38. 2/8 builtin/checkout.c: complete parallel checkout supportMatheus Tavares, May 4, 2021
  39. Derrick StoleeMay 5, 2021
  40. 3/8 checkout-index: add parallel checkout supportMatheus Tavares, May 4, 2021
  41. 4/8 parallel-checkout: add tests for basic operationsMatheus Tavares, May 4, 2021
  42. AIX failures on parallel checkout (new in v2.32.0-rc*)Ævar Arnfjörð Bjarmason, May 26, 2021
  43. Matheus Tavares BernardinoMay 26, 2021
  44. Junio C HamanoMay 26, 2021
  45. 5/8 parallel-checkout: add tests related to path collisionsMatheus Tavares, May 4, 2021
  46. 6/8 t0028: extract encoding helpers to lib-encoding.shMatheus Tavares, May 4, 2021
  47. 7/8 parallel-checkout: add tests related to .gitattributesMatheus Tavares, May 4, 2021
  48. 8/8 ci: run test round with parallel-checkout enabledMatheus Tavares, May 4, 2021
  49. Derrick StoleeMay 5, 2021
  50. Junio C HamanoMay 6, 2021

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.