git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [PATCH 0/5] Number truncation with 4+ GB files on 32-bit systems

From
Duy Nguyen <pclouds@gmail.com>
Date
Jul 13, 2016, 06:01 UTC
Message-ID
<CACsJy8CXg_x_4GRe88yUBYFm-_1=REAox=wK5yzu4W7Mf2qwgQ@mail.gmail.com>
In-Reply-To
<xmqq60sa4mnv.fsf@gitster.mtv.corp.google.com>
On Tue, Jul 12, 2016 at 10:38 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 16 quoted lines
> Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:
>
>> Since I now could reproduce the problem that Christoph showed, I
>> decided to send the good patches out. To sum up, we use "unsigned
>> long" in some places related to file size. On 32-bit systems, it's
>> limited to 32 bits even though the system can handle files larger than
>> that (off_t is 64-bit). This fixes it.
>>
>> clang -Wshorten-64-to-32 is very helpful to spot these problems. I
>> have a couple more patches to clean all these warnings, but some need
>> more code study to see what is the right way to do.
>>
>> Most of the rest seems harmless, except for the local variable "size"
>> in builtin/pack-objects.c:write_one(). I might send 6/5 for that one.
>
> With 1-7/5 it appears t1050 is broken?

Oops, sorry I focused on testing big repos and forgot the test suite. But hehe.. it's a good thing. That test was added by 735efde, where fsck could keep on going after failing to inflate some blobs (instead of hitting malloc failure and abort). Now it's even better, fsck passes. I'll fix that up and resend with your squashes in.

-- 
Duy
Previous: Junio C HamanoNext: Nguyễn Thái Ngọc Duy
Message 28 of 39 in “[bug] Reliably Reproducible Bad Packing of Objects”
  1. Christoph MichelbachJun 24, 2016
  2. Duy NguyenJul 2, 2016
  3. Duy NguyenJul 2, 2016
  4. 0/5 Number truncation with 4+ GB files on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 5, 2016
  5. 1/5 pack-objects: pass length to check_pack_crc() without truncationNguyễn Thái Ngọc Duy, Jul 5, 2016
  6. Junio C HamanoJul 12, 2016
  7. 5/5 index-pack: correct "offset" type in unpack_entry_data()Nguyễn Thái Ngọc Duy, Jul 5, 2016
  8. 4/5 index-pack: report correct bad object offsets even if they are largeNguyễn Thái Ngọc Duy, Jul 5, 2016
  9. Junio C HamanoJul 12, 2016
  10. Junio C HamanoJul 12, 2016
  11. 2/5 sha1_file.c: use type off_t* for object_info->disk_sizepNguyễn Thái Ngọc Duy, Jul 5, 2016
  12. Junio C HamanoJul 12, 2016
  13. Junio C HamanoJul 12, 2016
  14. 3/5 index-pack: correct "len" type in unpack_data()Nguyễn Thái Ngọc Duy, Jul 5, 2016
  15. Johannes SixtJul 5, 2016
  16. Duy NguyenJul 6, 2016
  17. Junio C HamanoJul 6, 2016
  18. Junio C HamanoJul 6, 2016
  19. Christoph MichelbachJul 5, 2016
  20. Duy NguyenJul 10, 2016
  21. 6/5 pack-objects: do not truncate result in-pack object size on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 10, 2016
  22. 7/5 fsck: use streaming interface for large blobs in packNguyễn Thái Ngọc Duy, Jul 10, 2016
  23. Junio C HamanoJul 12, 2016
  24. Junio C HamanoJul 12, 2016
  25. Junio C HamanoJul 12, 2016
  26. Junio C HamanoJul 12, 2016
  27. Junio C HamanoJul 12, 2016
  28. Duy NguyenJul 13, 2016
  29. 0/7 Number truncation with 4+ GB files on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 13, 2016
  30. 4/7 index-pack: report correct bad object offsets even if they are largeNguyễn Thái Ngọc Duy, Jul 13, 2016
  31. 3/7 index-pack: correct "len" type in unpack_data()Nguyễn Thái Ngọc Duy, Jul 13, 2016
  32. 2/7 sha1_file.c: use type off_t* for object_info->disk_sizepNguyễn Thái Ngọc Duy, Jul 13, 2016
  33. 5/7 index-pack: correct "offset" type in unpack_entry_data()Nguyễn Thái Ngọc Duy, Jul 13, 2016
  34. 7/7 fsck: use streaming interface for large blobs in packNguyễn Thái Ngọc Duy, Jul 13, 2016
  35. 6/7 pack-objects: do not truncate result in-pack object size on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 13, 2016
  36. 1/7 pack-objects: pass length to check_pack_crc() without truncationNguyễn Thái Ngọc Duy, Jul 13, 2016
  37. Junio C HamanoJul 13, 2016
  38. Christoph MichelbachJul 6, 2016
  39. Duy NguyenJul 6, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.