git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 01/11] index-pack, unpack-objects: use size_t for object size

From
Torsten Bögershausen <tboegi@web.de>
Date
May 10, 2026, 09:14 UTC
Message-ID
<20260510091436.GA5880@tb-raspi4>
In-Reply-To
<xmqqik8w9eb8.fsf@gitster.g>
Show 61 quoted lines
> 
> Hmph, I do not think there is "up-cast" to keep.  And we do not
> dereference a random pointer that would be suitable for unsigned
> char * as if it were "unsigned long *" or "size_t *" in this code.
> 
> This came from commit 48fb7deb5bbd87933e7d314b73d7c1b52667f80f
> 
> Author: Linus Torvalds <torvalds@linux-foundation.org>
> Date:   Wed Jun 17 17:22:27 2009 -0700
> 
>     Fix big left-shifts of unsigned char
>     
>     Shifting 'unsigned char' or 'unsigned short' left can result in sign
>     extension errors, since the C integer promotion rules means that the
>     unsigned char/short will get implicitly promoted to a signed 'int' due to
>     the shift (or due to other operations).
>     
>     This normally doesn't matter, but if you shift things up sufficiently, it
>     will now set the sign bit in 'int', and a subsequent cast to a bigger type
>     (eg 'long' or 'unsigned long') will now sign-extend the value despite the
>     original expression being unsigned.
>     
>     One example of this would be something like
>     
>             unsigned long size;
>             unsigned char c;
>     
>             size += c << 24;
>     
>     where despite all the variables being unsigned, 'c << 24' ends up being a
>     signed entity, and will get sign-extended when then doing the addition in
>     an 'unsigned long' type.
> 
> 
> You could rewrite Linus's example to
> 
> 	unsigned char *cp;
> 	unsigned long size;
> 	unsigned char c;
> 
> 	c = *cp;
> 	size += ((unsigned long)c) << 24;
> 
> While I am sympathetic to that position, I also would not mind
> 
> 	unsigned char *cp;
> 	unsigned long size;
> 	unsigned long c;
> 
> 	c = *cp;
> 	size += c << 24;
> 
> all that much.  In any case, such a "clean-up" has little to do with
> the topic under discussion, and itshould be discussed separately on
> its own merit, most likely when the dust settles after this topic
> lands.  Let's not contaminate the patches that is "a trivial rewrite
> that is so obviously correct to fix the assumption that ulong and
> size_t are of the same size everywhere" with unrelated clean-up.
> 
> Thanks.
> 

Sorry for the confusion and noise. My brain insisted to read c = *cp; // fetch 8 bits from memory, upcast to unsigned long as if we have written c = *(long*)cp; // fetch 32/64 bits from memory which is a completely different thing.

In short: all is good. Thanks for digging and the patience.

Previous: Junio C HamanoNext: Johannes Schindelin via GitGitGadget
Message 23 of 60 in “Handle cloning of objects larger than 4GB on Windows”
  1. 0/6 Handle cloning of objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, Apr 28, 2026
  2. 1/6 index-pack, unpack-objects: use size_t for object sizeJohannes Schindelin via GitGitGadget, Apr 28, 2026
  3. Torsten BögershausenApr 30, 2026
  4. Johannes SchindelinMay 3, 2026
  5. 2/6 git-zlib: handle data streams larger than 4GBJohannes Schindelin via GitGitGadget, Apr 28, 2026
  6. 3/6 odb, packfile: use size_t for streaming object sizesJohannes Schindelin via GitGitGadget, Apr 28, 2026
  7. 4/6 delta, packfile: use size_t for delta header sizesJohannes Schindelin via GitGitGadget, Apr 28, 2026
  8. Derrick StoleeApr 29, 2026
  9. Johannes SchindelinMay 3, 2026
  10. 5/6 test-tool: add a helper to synthesize large packfilesJohannes Schindelin via GitGitGadget, Apr 28, 2026
  11. 6/6 t5608: add regression test for >4GB object cloneJohannes Schindelin via GitGitGadget, Apr 28, 2026
  12. Derrick StoleeApr 29, 2026
  13. Jeff KingMay 1, 2026
  14. Derrick StoleeMay 1, 2026
  15. Johannes SchindelinMay 4, 2026
  16. Derrick StoleeApr 29, 2026
  17. 00/11 Handle cloning of objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, May 4, 2026
  18. 01/11 index-pack, unpack-objects: use size_t for object sizeJohannes Schindelin via GitGitGadget, May 4, 2026
  19. Torsten BögershausenMay 5, 2026
  20. Johannes SchindelinMay 8, 2026
  21. Torsten BögershausenMay 8, 2026
  22. Junio C HamanoMay 10, 2026
  23. Torsten BögershausenMay 10, 2026
  24. 02/11 git-zlib: handle data streams larger than 4GBJohannes Schindelin via GitGitGadget, May 4, 2026
  25. 03/11 odb, packfile: use size_t for streaming object sizesJohannes Schindelin via GitGitGadget, May 4, 2026
  26. Torsten BögershausenMay 5, 2026
  27. Johannes SchindelinMay 8, 2026
  28. 04/11 delta, packfile: use size_t for delta header sizesJohannes Schindelin via GitGitGadget, May 4, 2026
  29. 05/11 test-tool: add a helper to synthesize large packfilesJohannes Schindelin via GitGitGadget, May 4, 2026
  30. 06/11 t5608: add regression test for >4GB object cloneJohannes Schindelin via GitGitGadget, May 4, 2026
  31. 07/11 test-tool synthesize: use the unsafe hash for speedJohannes Schindelin via GitGitGadget, May 4, 2026
  32. 08/11 test-tool synthesize: precompute pack for 4 GiB + 1Johannes Schindelin via GitGitGadget, May 4, 2026
  33. Derrick StoleeMay 4, 2026
  34. Johannes SchindelinMay 5, 2026
  35. 09/11 test-tool synthesize: add precomputed SHA-256 pack for 4 GiB + 1Johannes Schindelin via GitGitGadget, May 4, 2026
  36. 10/11 t5608: mark >4GB tests as EXPENSIVEJohannes Schindelin via GitGitGadget, May 4, 2026
  37. 11/11 ci: run expensive tests on push builds to integration branchesJohannes Schindelin via GitGitGadget, May 4, 2026
  38. Derrick StoleeMay 4, 2026
  39. Junio C HamanoMay 5, 2026
  40. Junio C HamanoMay 5, 2026
  41. Johannes SchindelinMay 6, 2026
  42. Junio C HamanoMay 7, 2026
  43. Patrick SteinhardtMay 7, 2026
  44. Junio C HamanoMay 8, 2026
  45. 00/11 Handle cloning of objects larger than 4GB on WindowsJohannes Schindelin via GitGitGadget, May 8, 2026
  46. 01/11 index-pack, unpack-objects: use size_t for object sizeJohannes Schindelin via GitGitGadget, May 8, 2026
  47. 02/11 git-zlib: handle data streams larger than 4GBJohannes Schindelin via GitGitGadget, May 8, 2026
  48. 03/11 odb, packfile: use size_t for streaming object sizesJohannes Schindelin via GitGitGadget, May 8, 2026
  49. 04/11 delta, packfile: use size_t for delta header sizesJohannes Schindelin via GitGitGadget, May 8, 2026
  50. 05/11 test-tool: add a helper to synthesize large packfilesJohannes Schindelin via GitGitGadget, May 8, 2026
  51. 06/11 t5608: add regression test for >4GB object cloneJohannes Schindelin via GitGitGadget, May 8, 2026
  52. 07/11 test-tool synthesize: use the unsafe hash for speedJohannes Schindelin via GitGitGadget, May 8, 2026
  53. 08/11 test-tool synthesize: precompute pack for 4 GiB + 1Johannes Schindelin via GitGitGadget, May 8, 2026
  54. 09/11 test-tool synthesize: add precomputed SHA-256 pack for 4 GiB + 1Johannes Schindelin via GitGitGadget, May 8, 2026
  55. 10/11 t5608: mark >4GB tests as EXPENSIVEJohannes Schindelin via GitGitGadget, May 8, 2026
  56. 11/11 ci: run expensive tests on push builds to integration branchesJohannes Schindelin via GitGitGadget, May 8, 2026
  57. ci: enable EXPENSIVE for contributor buildsJunio C Hamano, May 10, 2026
  58. Patrick SteinhardtMay 11, 2026
  59. Junio C HamanoMay 11, 2026
  60. Patrick SteinhardtMay 11, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.