git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 6/6] t5608: add regression test for >4GB object clone

From
Jeff King <peff@peff.net>
Date
May 1, 2026, 06:38 UTC
Message-ID
<20260501063805.GA2038915@coredump.intra.peff.net>
In-Reply-To
<e1e8837f-7374-4079-ba87-ab95dd156e33@gmail.com>
On Wed, Apr 29, 2026 at 09:34:21AM -0400, Derrick Stolee wrote:
> As mentioned in the previous patch, constructing this large packfile
> takes ~4 minutes in CI pipelines. That's quite a lot to handle for
> every CI run.

And for local runs, too. ;) The test suite takes less than 90 seconds to run on my laptop, but t5608 by itself 160 seconds. Even if running it in parallel didn't slow down the rest of the suite (which is not true, because it's obviously hogging a whole processor the whole time), that's still almost doubling the run-time.

I'd also worry about assuming that the trash directory can hold 4+GB (maybe 8GB+ since we clone it?), especially since many of us use ram disks.

That said...
Show 7 quoted lines
> > +test_expect_success SIZE_T_IS_64BIT 'set up repo with >4GB object' '
> 
> Your prereq here prevents it from running on 32-bit builds, which is
> good. However, I wonder if it would be worth also specifying these
> tests as expensive. It's less likely that these layers will be touched
> often, so it should be enough to run these on major occasions, such as
> testing a release candidate.

I think it is already skipped in most cases, because t5608 requires the GIT_TEST_CLONE_2GB environment variable be set. Arguably it should just be using EXPENSIVE, too, as I do not think there is much value in having individual flags for all of the expensive tests. I think that test just predates the modern prereq system entirely.

> I suppose this also is a question for Junio and our process for
> validating releases. Do we have a certain cadence where we run the
> expensive tests? What has been our threshold for hiding a test case
> behind the expensive label?

AFAIK the labeling of expensive things is mostly ad-hoc, and nobody is systematically running them. Likewise for the t/perf tests, which are super expensive but do (very occasionally) turn up interesting regressions.

-Peff
Previous: Derrick StoleeNext: Derrick Stolee
Message 13 of 60 in “Handle cloning of objects larger than 4GB on Windows”
  1. 0/6 Handle cloning of objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, Apr 28, 2026
  2. 1/6 index-pack, unpack-objects: use size_t for object sizeJohannes Schindelin via GitGitGadget, Apr 28, 2026
  3. Torsten BögershausenApr 30, 2026
  4. Johannes SchindelinMay 3, 2026
  5. 2/6 git-zlib: handle data streams larger than 4GBJohannes Schindelin via GitGitGadget, Apr 28, 2026
  6. 3/6 odb, packfile: use size_t for streaming object sizesJohannes Schindelin via GitGitGadget, Apr 28, 2026
  7. 4/6 delta, packfile: use size_t for delta header sizesJohannes Schindelin via GitGitGadget, Apr 28, 2026
  8. Derrick StoleeApr 29, 2026
  9. Johannes SchindelinMay 3, 2026
  10. 5/6 test-tool: add a helper to synthesize large packfilesJohannes Schindelin via GitGitGadget, Apr 28, 2026
  11. 6/6 t5608: add regression test for >4GB object cloneJohannes Schindelin via GitGitGadget, Apr 28, 2026
  12. Derrick StoleeApr 29, 2026
  13. Jeff KingMay 1, 2026
  14. Derrick StoleeMay 1, 2026
  15. Johannes SchindelinMay 4, 2026
  16. Derrick StoleeApr 29, 2026
  17. 00/11 Handle cloning of objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, May 4, 2026
  18. 01/11 index-pack, unpack-objects: use size_t for object sizeJohannes Schindelin via GitGitGadget, May 4, 2026
  19. Torsten BögershausenMay 5, 2026
  20. Johannes SchindelinMay 8, 2026
  21. Torsten BögershausenMay 8, 2026
  22. Junio C HamanoMay 10, 2026
  23. Torsten BögershausenMay 10, 2026
  24. 02/11 git-zlib: handle data streams larger than 4GBJohannes Schindelin via GitGitGadget, May 4, 2026
  25. 03/11 odb, packfile: use size_t for streaming object sizesJohannes Schindelin via GitGitGadget, May 4, 2026
  26. Torsten BögershausenMay 5, 2026
  27. Johannes SchindelinMay 8, 2026
  28. 04/11 delta, packfile: use size_t for delta header sizesJohannes Schindelin via GitGitGadget, May 4, 2026
  29. 05/11 test-tool: add a helper to synthesize large packfilesJohannes Schindelin via GitGitGadget, May 4, 2026
  30. 06/11 t5608: add regression test for >4GB object cloneJohannes Schindelin via GitGitGadget, May 4, 2026
  31. 07/11 test-tool synthesize: use the unsafe hash for speedJohannes Schindelin via GitGitGadget, May 4, 2026
  32. 08/11 test-tool synthesize: precompute pack for 4 GiB + 1Johannes Schindelin via GitGitGadget, May 4, 2026
  33. Derrick StoleeMay 4, 2026
  34. Johannes SchindelinMay 5, 2026
  35. 09/11 test-tool synthesize: add precomputed SHA-256 pack for 4 GiB + 1Johannes Schindelin via GitGitGadget, May 4, 2026
  36. 10/11 t5608: mark >4GB tests as EXPENSIVEJohannes Schindelin via GitGitGadget, May 4, 2026
  37. 11/11 ci: run expensive tests on push builds to integration branchesJohannes Schindelin via GitGitGadget, May 4, 2026
  38. Derrick StoleeMay 4, 2026
  39. Junio C HamanoMay 5, 2026
  40. Junio C HamanoMay 5, 2026
  41. Johannes SchindelinMay 6, 2026
  42. Junio C HamanoMay 7, 2026
  43. Patrick SteinhardtMay 7, 2026
  44. Junio C HamanoMay 8, 2026
  45. 00/11 Handle cloning of objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, May 8, 2026
  46. 01/11 index-pack, unpack-objects: use size_t for object sizeJohannes Schindelin via GitGitGadget, May 8, 2026
  47. 02/11 git-zlib: handle data streams larger than 4GBJohannes Schindelin via GitGitGadget, May 8, 2026
  48. 03/11 odb, packfile: use size_t for streaming object sizesJohannes Schindelin via GitGitGadget, May 8, 2026
  49. 04/11 delta, packfile: use size_t for delta header sizesJohannes Schindelin via GitGitGadget, May 8, 2026
  50. 05/11 test-tool: add a helper to synthesize large packfilesJohannes Schindelin via GitGitGadget, May 8, 2026
  51. 06/11 t5608: add regression test for >4GB object cloneJohannes Schindelin via GitGitGadget, May 8, 2026
  52. 07/11 test-tool synthesize: use the unsafe hash for speedJohannes Schindelin via GitGitGadget, May 8, 2026
  53. 08/11 test-tool synthesize: precompute pack for 4 GiB + 1Johannes Schindelin via GitGitGadget, May 8, 2026
  54. 09/11 test-tool synthesize: add precomputed SHA-256 pack for 4 GiB + 1Johannes Schindelin via GitGitGadget, May 8, 2026
  55. 10/11 t5608: mark >4GB tests as EXPENSIVEJohannes Schindelin via GitGitGadget, May 8, 2026
  56. 11/11 ci: run expensive tests on push builds to integration branchesJohannes Schindelin via GitGitGadget, May 8, 2026
  57. ci: enable EXPENSIVE for contributor buildsJunio C Hamano, May 10, 2026
  58. Patrick SteinhardtMay 11, 2026
  59. Junio C HamanoMay 11, 2026
  60. Patrick SteinhardtMay 11, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.