git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH v2 07/12] packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`

From
Junio C Hamano <gitster@pobox.com>
Date
Aug 7, 2026, 22:06 UTC
Message-ID
<xmqqcxvtd1rk.fsf@gitster.g>
In-Reply-To
<ca928b457959ab8bfa643c65f83ca1ec4289fdd3.1785946479.git.gitgitgadget@gmail.com>
[jc: Sorry, I hit <SEND> before I was ready]

"Johannes Schindelin via GitGitGadget" <gitgitgadget@gmail.com> writes:

Show 15 quoted lines
> diff --git a/git-zlib.h b/git-zlib.h
> index 44380e8ad3..0b24b15bd0 100644
> --- a/git-zlib.h
> +++ b/git-zlib.h
> @@ -5,8 +5,8 @@
>  
>  typedef struct git_zstream {
>  	struct z_stream_s z;
> -	unsigned long avail_in;
> -	unsigned long avail_out;
> +	size_t avail_in;
> +	size_t avail_out;
>  	size_t total_in;
>  	size_t total_out;
>  	unsigned char *next_in;

We use 'size_t', which means we can use a buffer larger than 4 GB on systems where 'size_t' is wider than a 32-bit 'unsigned long'. But these represent the size of a single contiguous buffer, so I think that is why the log message mentioned that this is more about type consistency than being able to handle larger data, as I do not think anyone would reasonably feed a contiguous buffer larger than 4 GB in one go in practice. For that reason, two details stood out to me:

 - zlib_buf_cap() still returns 'unsigned long', and
   zlib_pre_call() feeds these potentially wider values to it.
   Is it possible that we trigger truncation before 'avail_in'
   or 'avail_out' is compared with 'ZLIB_BUF_MAX' in
   zlib_buf_cap()?
 - unpack_object_header_buffer() still takes an 'unsigned long'
   length, while oe_get_size_slow() in 'builtin/pack-objects.c'
   passes a 'size_t' 'avail' to it.  This comes from use_pack(),
   so it is a relatively small value stored in a wider 'size_t',
   but I am unsure whether your static checker would flag this for
   potential truncation.

They are probably harmless in practice, but they are still a bit concerning from the standpoint of type consistency.

Previous: Junio C HamanoNext: Johannes Schindelin via GitGitGadget
Message 32 of 55 in “Next size_t stop: pack-objects/delta”
  1. 00/12 Next size_t stop: pack-objects/deltaJohannes Schindelin via GitGitGadget, Jul 9, 2026
  2. 01/12 diff-delta: widen `struct delta_index`' size fields to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  3. Patrick SteinhardtAug 5, 2026
  4. Johannes SchindelinAug 5, 2026
  5. 02/12 delta: widen `create_delta_index()` parameter to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  6. 03/12 pack-objects: widen delta-cache accounting to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  7. Patrick SteinhardtAug 5, 2026
  8. Johannes SchindelinAug 5, 2026
  9. 04/12 pack-objects: widen `free_unpacked()` return to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  10. Patrick SteinhardtAug 5, 2026
  11. 05/12 pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  12. 06/12 delta: widen `create_delta()` and `diff_delta()` to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  13. Patrick SteinhardtAug 5, 2026
  14. 07/12 packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  15. 08/12 archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  16. 09/12 diff: widen `deflate_it()`'s bound local from int to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  17. 10/12 http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  18. 11/12 t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  19. 12/12 git-zlib: widen `git_deflate_bound()` to `size_t`Johannes Schindelin via GitGitGadget, Jul 9, 2026
  20. Patrick SteinhardtAug 5, 2026
  21. Johannes SchindelinAug 5, 2026
  22. 00/12 Next size_t stop: pack-objects/deltaJohannes Schindelin via GitGitGadget, Aug 5, 2026
  23. 02/12 delta: widen `create_delta_index()` parameter to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  24. 01/12 diff-delta: widen `struct delta_index`' size fields to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  25. Patrick SteinhardtAug 6, 2026
  26. 03/12 pack-objects: widen delta-cache accounting to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  27. 05/12 pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  28. 04/12 pack-objects: widen `free_unpacked()` return to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  29. 06/12 delta: widen `create_delta()` and `diff_delta()` to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  30. 07/12 packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  31. Junio C HamanoAug 7, 2026
  32. Junio C HamanoAug 7, 2026
  33. 09/12 diff: widen `deflate_it()`'s bound local from int to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  34. 08/12 archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  35. 11/12 t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  36. 10/12 http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  37. 12/12 git-zlib: widen `git_deflate_bound()` to `size_t`Johannes Schindelin via GitGitGadget, Aug 5, 2026
  38. Patrick SteinhardtAug 6, 2026
  39. Junio C HamanoAug 6, 2026
  40. Junio C HamanoAug 10, 2026
  41. 00/13 Next size_t stop: pack-objects/deltaJohannes Schindelin via GitGitGadget, Aug 13, 2026
  42. 01/13 diff-delta: widen `struct delta_index`' size fields to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  43. 02/13 delta: widen `create_delta_index()` parameter to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  44. 03/13 pack-objects: widen delta-cache accounting to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  45. 04/13 pack-objects: widen `free_unpacked()` return to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  46. 05/13 pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  47. 07/13 packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  48. 06/13 delta: widen `create_delta()` and `diff_delta()` to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  49. 08/13 archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  50. 09/13 diff: widen `deflate_it()`'s bound local from int to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  51. 10/13 http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  52. 12/13 git-zlib: widen `git_deflate_bound()` to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  53. 13/13 packfile: widen `unpack_object_header_buffer()` to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  54. 11/13 t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`Johannes Schindelin via GitGitGadget, Aug 13, 2026
  55. Junio C HamanoAug 13, 2026

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.