git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v3 3/9] t4141: fix inefficient use of dd(1)

From
Patrick Steinhardt <ps@pks.im>
Date
Jul 6, 2026, 06:23 UTC
Message-ID
<20260706-b4-pks-t-fixes-for-GIT-TEST-LONG-v3-3-4f6c5a37fd1f@pks.im>
In-Reply-To
<20260706-b4-pks-t-fixes-for-GIT-TEST-LONG-v3-0-4f6c5a37fd1f@pks.im>

In t4141 we generate a patch that is roughly 1GB in size to verify that git-apply(1) indeed rejects that patch. We generate that patch by prepending a patch header and then executing `test-tool genzeros` without a limit. This causes us to print infinitely many zeros, and we limit the overall amount of generated bytes via `test_copy_bytes`.

This test setup is extremely expensive, as `test_copy_bytes` is implemented via `dd ibs=1 count="$1"`, which copies data one byte at a time. So as we write 1GB of data, we end up doing 1 billion reads and writes. This naturally takes a while: it takes 6 minutes on my system, and around 40 minutes in some CI jobs!

We can do much better though, as genzeros already knows to handle an optional limit of how much data it is supposed to write, which allows us to remove the call to `test_copy_bytes`. Furthermore, it has already been optimized to generate the data fast.

And indeed, doing this conversion drops the test execution to less than a second on my machine. That means that in theory it becomes feasible to drop the EXPENSIVE prerequisite now. But git-apply(1) still soaks up 1GB of data into memory, which may count as being expensive. Consequently, we keep the prerequisite intact.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t4141-apply-too-large.sh | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/t/t4141-apply-too-large.sh b/t/t4141-apply-too-large.sh
index eac6f7e151..9dbed940db 100755
--- a/t/t4141-apply-too-large.sh
+++ b/t/t4141-apply-too-large.sh
@@ -5,7 +5,6 @@ test_description='git apply with too-large patch'
 . ./test-lib.sh
 
 test_expect_success EXPENSIVE 'git apply rejects patches that are too large' '
-	sz=$((1024 * 1024 * 1023)) &&
 	{
 		cat <<-\EOF &&
 		diff --git a/file b/file
@@ -14,8 +13,8 @@ test_expect_success EXPENSIVE 'git apply rejects patches that are too large' '
 		+++ b/file
 		@@ -0,0 +1 @@
 		EOF
-		test-tool genzeros
-	} | test_copy_bytes $sz | test_must_fail git apply 2>err &&
+		test-tool genzeros $((1024 * 1024 * 1023))
+	} | test_must_fail git apply 2>err &&
 	grep "patch too large" err
 '
 
-- 
2.55.0.795.g602f6c329a.dirty
Previous: Patrick SteinhardtNext: Patrick Steinhardt
Message 4 of 12 in “t: fixes and improvements for GIT_TEST_LONG”
  1. 0/9 t: fixes and improvements for GIT_TEST_LONGPatrick Steinhardt, Jul 6, 2026
  2. 1/9 README: add GitLab CI badge to make it more discoverablePatrick Steinhardt, Jul 6, 2026
  3. 2/9 t0021: skip EXPENSIVE test that is broken without SIZE_T_IS_64BITPatrick Steinhardt, Jul 6, 2026
  4. 3/9 t4141: fix inefficient use of dd(1)Patrick Steinhardt, Jul 6, 2026
  5. 4/9 t5608: reduce maximum disk usagePatrick Steinhardt, Jul 6, 2026
  6. 5/9 t7508: skip EXPENSIVE test that is broken without SIZE_T_IS_64BITPatrick Steinhardt, Jul 6, 2026
  7. 6/9 t7900: clean up large EXPENSIVE repositoryPatrick Steinhardt, Jul 6, 2026
  8. 7/9 t: use `test_bool_env` to parse GIT_TEST_LONGPatrick Steinhardt, Jul 6, 2026
  9. 8/9 gitlab-ci: disable RAM disk on macOS jobsPatrick Steinhardt, Jul 6, 2026
  10. 9/9 gitlab-ci: enable "GIT_TEST_LONG"Patrick Steinhardt, Jul 6, 2026
  11. Junio C HamanoJul 6, 2026
  12. Jeff KingJul 7, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.