git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 2/6] object-file.c: use size_t for header lengths

From
Johannes Schindelin <johannes.schindelin@gmx.de>
Date
Jun 16, 2026, 14:48 UTC
Message-ID
<29ae7ae4-4ae7-2057-3273-d5024a893d2e@gmx.de>
In-Reply-To
<ai-5XO9gsc_HdMFX@pks.im>
Hi Patrick,
On Mon, 15 Jun 2026, Patrick Steinhardt wrote:
Show 16 quoted lines
> On Thu, Jun 04, 2026 at 05:15:08PM +0000, Philip Oakley via GitGitGadget wrote:
> > From: Philip Oakley <philipoakley@iee.email>
> > 
> > Continue walking the code path for the >4GB `hash-object --literally`
> > test. The `hash_object_file_literally()` function internally uses both
> > `hash_object_file()` and `write_object_file_prepare()`. Both function
> > signatures use `unsigned long` rather than `size_t` for the mem buffer
> > sizes. Use `size_t` instead, for LLP64 compatibility.
> > 
> > While at it, convert those function's object's header buffer length to
> > `size_t` for consistency. The value is already upcast to `uintmax_t` for
> > print format compatibility.
> 
> One thing I was wondering is whether we should rather migrate to a size
> that is consistent across different platforms. We could e.g. `typedef
> uint64_t objsize_t` and then use that going forward.

No, the point of `size_t` is to represent what the current platform can handle in-memory. That cannot (and should not) be consolidated.

> I guess the question though is whether that'd buy us anything. In other
> words, are there any platforms that we care about where `size_t` is only
> 32 bit wide? And would such platforms even be able to handle such large
> objects?

There are ways to handle objects larger than 4GB on 32-bit platforms, via streaming. In those cases, what you need is `off_t`, not `size_t`.

Obviously, there is a large class of problems with such setups. For example, you can forget about efficiently reconstructing a large Git object from a delta chain. If you cannot do that in-memory, trying to work around that limitation merely opens up the user for a world of pain.

Ciao, Johannes

Previous: Patrick SteinhardtNext: Philip Oakley via GitGitGadget
Message 5 of 27 in “Support hashing objects larger than 4GB on Windows”
  1. 0/6 Support hashing objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, Jun 4, 2026
  2. 1/6 hash-object: demonstrate a >4GB/LLP64 problemPhilip Oakley via GitGitGadget, Jun 4, 2026
  3. 2/6 object-file.c: use size_t for header lengthsPhilip Oakley via GitGitGadget, Jun 4, 2026
  4. Patrick SteinhardtJun 15, 2026
  5. Johannes SchindelinJun 16, 2026
  6. 3/6 hash algorithms: use size_t for section lengthsPhilip Oakley via GitGitGadget, Jun 4, 2026
  7. Patrick SteinhardtJun 15, 2026
  8. Johannes SchindelinJun 16, 2026
  9. 4/6 hash-object --stdin: verify that it works with >4GB/LLP64Philip Oakley via GitGitGadget, Jun 4, 2026
  10. Patrick SteinhardtJun 15, 2026
  11. 5/6 hash-object: add another >4GB/LLP64 test casePhilip Oakley via GitGitGadget, Jun 4, 2026
  12. Patrick SteinhardtJun 15, 2026
  13. Johannes SchindelinJun 16, 2026
  14. 6/6 hash-object: add a >4GB/LLP64 test case using filtered inputPhilip Oakley via GitGitGadget, Jun 4, 2026
  15. Philip OakleyJun 4, 2026
  16. Junio C HamanoJun 8, 2026
  17. 0/6 Support hashing objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, Jun 16, 2026
  18. 1/6 hash-object: demonstrate a >4GB/LLP64 problemPhilip Oakley via GitGitGadget, Jun 16, 2026
  19. 2/6 object-file.c: use size_t for header lengthsPhilip Oakley via GitGitGadget, Jun 16, 2026
  20. 3/6 hash algorithms: use size_t for section lengthsPhilip Oakley via GitGitGadget, Jun 16, 2026
  21. 4/6 hash-object --stdin: verify that it works with >4GB/LLP64Philip Oakley via GitGitGadget, Jun 16, 2026
  22. 5/6 hash-object: add another >4GB/LLP64 test casePhilip Oakley via GitGitGadget, Jun 16, 2026
  23. 6/6 hash-object: add a >4GB/LLP64 test case using filtered inputPhilip Oakley via GitGitGadget, Jun 16, 2026
  24. Junio C HamanoJun 16, 2026
  25. How does GitGitGadget generate range-diffs, was Re: [PATCH v2 0/6] Support hashing objects larger than 4GB on WindowsJohannes Schindelin, Jun 17, 2026
  26. Junio C HamanoJun 17, 2026
  27. Patrick SteinhardtJun 18, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.