git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 1/6] hash-object: demonstrate a >4GB/LLP64 problem

From
Philip Oakley via GitGitGadget <gitgitgadget@gmail.com>
Date
Jun 16, 2026, 14:49 UTC
Message-ID
<9c01bac407034c6361dc54af7f11f64e8af7db0e.1781621398.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.2138.v2.git.1781621398.gitgitgadget@gmail.com>
From: Philip Oakley <philipoakley@iee.email>

On LLP64 systems, such as Windows, the size of `long`, `int`, etc. is only 32 bits (for backward compatibility). Git's use of `unsigned long` for file memory sizes in many places, rather than size_t, limits the handling of large files on LLP64 systems (commonly given as `>4GB`).

Provide a minimum test for handling a >4GB file. The `hash-object` command, with the `--literally` and without `-w` option avoids writing the object, either loose or packed. This avoids the code paths hitting the `bigFileThreshold` config test code, the zlib code, and the pack code.

Subsequent patches will walk the test's call chain, converting types to `size_t` (which is larger in LLP64 data models) where appropriate.

Signed-off-by: Philip Oakley <philipoakley@iee.email>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
 t/t1007-hash-object.sh | 11 +++++++++++
 1 file changed, 11 insertions(+)
diff --git a/t/t1007-hash-object.sh b/t/t1007-hash-object.sh
index de076293b6..7867fd1dbf 100755
--- a/t/t1007-hash-object.sh
+++ b/t/t1007-hash-object.sh
@@ -49,6 +49,9 @@ test_expect_success 'setup' '
 
 	example sha1:ddd3f836d3e3fbb7ae289aa9ae83536f76956399
 	example sha256:b44fe1fe65589848253737db859bd490453510719d7424daab03daf0767b85ae
+
+	large5GB sha1:0be2be10a4c8764f32c4bf372a98edc731a4b204
+	large5GB sha256:dc18ca621300c8d3cfa505a275641ebab00de189859e022a975056882d313e64
 	EOF
 '
 
@@ -258,4 +261,12 @@ test_expect_success '--stdin outside of repository (uses default hash)' '
 	test_cmp expect actual
 '
 
+test_expect_failure EXPENSIVE,SIZE_T_IS_64BIT,!LONG_IS_64BIT \
+		'files over 4GB hash literally' '
+	test-tool genzeros $((5*1024*1024*1024)) >big &&
+	test_oid large5GB >expect &&
+	git hash-object --stdin --literally <big >actual &&
+	test_cmp expect actual
+'
+
 test_done
-- 
gitgitgadget
Previous: Johannes Schindelin via GitGitGadgetNext: Philip Oakley via GitGitGadget
Message 18 of 27 in “Support hashing objects larger than 4GB on Windows”
  1. 0/6 Support hashing objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, Jun 4, 2026
  2. 1/6 hash-object: demonstrate a >4GB/LLP64 problemPhilip Oakley via GitGitGadget, Jun 4, 2026
  3. 2/6 object-file.c: use size_t for header lengthsPhilip Oakley via GitGitGadget, Jun 4, 2026
  4. Patrick SteinhardtJun 15, 2026
  5. Johannes SchindelinJun 16, 2026
  6. 3/6 hash algorithms: use size_t for section lengthsPhilip Oakley via GitGitGadget, Jun 4, 2026
  7. Patrick SteinhardtJun 15, 2026
  8. Johannes SchindelinJun 16, 2026
  9. 4/6 hash-object --stdin: verify that it works with >4GB/LLP64Philip Oakley via GitGitGadget, Jun 4, 2026
  10. Patrick SteinhardtJun 15, 2026
  11. 5/6 hash-object: add another >4GB/LLP64 test casePhilip Oakley via GitGitGadget, Jun 4, 2026
  12. Patrick SteinhardtJun 15, 2026
  13. Johannes SchindelinJun 16, 2026
  14. 6/6 hash-object: add a >4GB/LLP64 test case using filtered inputPhilip Oakley via GitGitGadget, Jun 4, 2026
  15. Philip OakleyJun 4, 2026
  16. Junio C HamanoJun 8, 2026
  17. 0/6 Support hashing objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, Jun 16, 2026
  18. 1/6 hash-object: demonstrate a >4GB/LLP64 problemPhilip Oakley via GitGitGadget, Jun 16, 2026
  19. 2/6 object-file.c: use size_t for header lengthsPhilip Oakley via GitGitGadget, Jun 16, 2026
  20. 3/6 hash algorithms: use size_t for section lengthsPhilip Oakley via GitGitGadget, Jun 16, 2026
  21. 4/6 hash-object --stdin: verify that it works with >4GB/LLP64Philip Oakley via GitGitGadget, Jun 16, 2026
  22. 5/6 hash-object: add another >4GB/LLP64 test casePhilip Oakley via GitGitGadget, Jun 16, 2026
  23. 6/6 hash-object: add a >4GB/LLP64 test case using filtered inputPhilip Oakley via GitGitGadget, Jun 16, 2026
  24. Junio C HamanoJun 16, 2026
  25. How does GitGitGadget generate range-diffs, was Re: [PATCH v2 0/6] Support hashing objects larger than 4GB on WindowsJohannes Schindelin, Jun 17, 2026
  26. Junio C HamanoJun 17, 2026
  27. Patrick SteinhardtJun 18, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.