git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 5/7] t0027: cache expected output files

From
Patrick Steinhardt <ps@pks.im>
Date
Nov 10, 2026, 06:53 UTC
Message-ID
<avK_XnoZ7Asakg25@pks.im>
In-Reply-To
<20261109144651.GA14738@tb-raspi4>
On Mon, Nov 09, 2026 at 03:46:51PM +0100, Torsten Bögershausen wrote:
Show 40 quoted lines
> On Mon, Nov 09, 2026 at 10:58:26AM +0100, Patrick Steinhardt wrote:
> > In the preceding commit we have optimized one aspect of t0027 already.
> > Another inefficiency is `compare_files ()` and `compare_ws_file ()`,
> > which are both called more than a 1000 times.
> > 
> > These functions will repeatedly generate the same expected outcome even
> > though there's only a small handful of input files that the expected
> > outcome is computed from. Recomputing those every time is of course
> > quite pointless.
> > 
> > Improve this by computing the expected outcome once, only, to reduce the
> > number of spawned processes. This reduces the runtime by around 20-25%,
> > and on Windows the runtime decreases from 480 to 380 seconds.
> > 
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> >  t/t0027-auto-crlf.sh | 17 +++++++++--------
> >  1 file changed, 9 insertions(+), 8 deletions(-)
> > 
> > diff --git a/t/t0027-auto-crlf.sh b/t/t0027-auto-crlf.sh
> > index 504a7ea54bb..b021f0855a8 100755
> > --- a/t/t0027-auto-crlf.sh
> > +++ b/t/t0027-auto-crlf.sh
> > @@ -5,22 +5,23 @@ test_description='CRLF conversion all combinations'
> >  . ./test-lib.sh
> >  
> >  compare_files () {
> > -	tr '\015\000' QN <"$1" >"$1".expect &&
> > +	{ test -f "$1".expect || tr '\015\000' QN <"$1" >"$1".expect; } &&
> >  	tr '\015\000' QN <"$2" | tr -d 'Z' >"$2".actual &&
> > -	test_cmp "$1".expect "$2".actual &&
> > -	rm "$1".expect "$2".actual
> > +	test_cmp "$1".expect "$2".actual
> >  }
> >  
> 
> Nice.
> If you want to save even more CPU/SSD-cycles,
> you can prepare the expect files like this:
> (No, I didn't do any benchmarks, nor look at ws_compare)

Thanks! I benchmarked this and it doesn't seem to make much of a difference. So I'd propose to stick with the simple change for now, if that's alright with you.

Patrick
Previous: Torsten BögershausenNext: Torsten Bögershausen
Message 28 of 40 in “ci: some speedups for slow tests”
  1. 0/7 ci: some speedups for slow testsPatrick Steinhardt, Nov 6, 2026
  2. 1/7 gitlab-ci: speed up installation of Git for Windows SDKPatrick Steinhardt, Nov 6, 2026
  3. Junio C HamanoNov 6, 2026
  4. Patrick SteinhardtNov 9, 2026
  5. 2/7 gitlab-ci: exclude intermediate outputs from Windows artifactsPatrick Steinhardt, Nov 6, 2026
  6. 3/7 ci: build optimized binaries with MesonPatrick Steinhardt, Nov 6, 2026
  7. Junio C HamanoNov 6, 2026
  8. 4/7 t0027: optimize how we reset repository statePatrick Steinhardt, Nov 6, 2026
  9. Junio C HamanoNov 6, 2026
  10. Torsten BögershausenNov 8, 2026
  11. Patrick SteinhardtNov 9, 2026
  12. 5/7 t0027: cache expected output filesPatrick Steinhardt, Nov 6, 2026
  13. 6/7 t1092: set up repositories from templatesPatrick Steinhardt, Nov 6, 2026
  14. 7/7 t5551: use reftables for repositories with many refsPatrick Steinhardt, Nov 6, 2026
  15. Junio C HamanoNov 6, 2026
  16. 0/7 ci: some speedups for slow testsPatrick Steinhardt, Nov 9, 2026
  17. 1/7 gitlab-ci: speed up installation of Git for Windows SDKPatrick Steinhardt, Nov 9, 2026
  18. Junio C HamanoNov 9, 2026
  19. Patrick SteinhardtNov 10, 2026
  20. Junio C HamanoNov 10, 2026
  21. 2/7 gitlab-ci: exclude intermediate outputs from Windows artifactsPatrick Steinhardt, Nov 9, 2026
  22. 3/7 ci: build optimized binaries with MesonPatrick Steinhardt, Nov 9, 2026
  23. 4/7 t0027: optimize how we reset repository statePatrick Steinhardt, Nov 9, 2026
  24. Junio C HamanoNov 9, 2026
  25. Patrick SteinhardtNov 10, 2026
  26. 5/7 t0027: cache expected output filesPatrick Steinhardt, Nov 9, 2026
  27. Torsten BögershausenNov 9, 2026
  28. Patrick SteinhardtNov 10, 2026
  29. Torsten BögershausenNov 10, 2026
  30. 6/7 t1092: set up repositories from templatesPatrick Steinhardt, Nov 9, 2026
  31. 7/7 t5551: use reftables for repositories with many refsPatrick Steinhardt, Nov 9, 2026
  32. Johannes SixtNov 12, 2026
  33. 0/7 ci: some speedups for slow testsPatrick Steinhardt, Nov 11, 2026
  34. 1/7 gitlab-ci: speed up installation of Git for Windows SDKPatrick Steinhardt, Nov 11, 2026
  35. 2/7 gitlab-ci: exclude intermediate outputs from Windows artifactsPatrick Steinhardt, Nov 11, 2026
  36. 3/7 ci: build optimized binaries with MesonPatrick Steinhardt, Nov 11, 2026
  37. 4/7 t0027: optimize how we reset repository statePatrick Steinhardt, Nov 11, 2026
  38. 5/7 t0027: cache expected output filesPatrick Steinhardt, Nov 11, 2026
  39. 6/7 t1092: set up repositories from templatesPatrick Steinhardt, Nov 11, 2026
  40. 7/7 t5551: use reftables for repositories with many refsPatrick Steinhardt, Nov 11, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.