git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: [bug] Reliably Reproducible Bad Packing of Objects

From
Duy Nguyen <pclouds@gmail.com>
Date
Jul 2, 2016, 14:35 UTC
Message-ID
<CACsJy8AJxbnEXciP91_r0jtH3mv+T_dqQZt9pKnJvSK-s9Yamg@mail.gmail.com>
In-Reply-To
<20160702091034.GA31852@duynguyen>
On Sat, Jul 2, 2016 at 11:10 AM, Duy Nguyen <pclouds@gmail.com> wrote:
Show 21 quoted lines
> On Sat, Jun 25, 2016 at 12:38:22AM +0200, Christoph Michelbach wrote:
>> Hi,
>>
>> when run on a 32 bit system (Linux system, don't know about other
>> systems),
>>
>> mkdir test && cd test && git init && touch someFile && git add someFile
>> && git commit -m "Initial commit." && dd if=/dev/urandom
>> of=bigBinaryFile bs=1MB count=4294 && git add bigBinaryFile && git
>> commit -m "Introduced big biary file."
>>
>> reliably produces this error message: "error: bad packed object CRC
>> for"
>
>
> I tried with a 32-bit git build on 64-bit system and did not see this
> message. Maybe it's because the 64-bit system. Just too lazy to bring
> up a 32-bit VM to process 4GB of data.
>
> But I did look around a bit, and the problem is probably number
> truncation. Could you try this patch?

Just FYI. There may be more truncation problems on 32-bit systems. "clang -Wshorten-64-to-32" reports a couple of spots that could be real problems (pack-objects, index-pack...) for handling 4+ GB files on these systems. Still working on it...

-- 
Duy
Previous: Duy NguyenNext: Nguyễn Thái Ngọc Duy
Message 3 of 39 in “[bug] Reliably Reproducible Bad Packing of Objects”
  1. Christoph MichelbachJun 24, 2016
  2. Duy NguyenJul 2, 2016
  3. Duy NguyenJul 2, 2016
  4. 0/5 Number truncation with 4+ GB files on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 5, 2016
  5. 1/5 pack-objects: pass length to check_pack_crc() without truncationNguyễn Thái Ngọc Duy, Jul 5, 2016
  6. Junio C HamanoJul 12, 2016
  7. 5/5 index-pack: correct "offset" type in unpack_entry_data()Nguyễn Thái Ngọc Duy, Jul 5, 2016
  8. 4/5 index-pack: report correct bad object offsets even if they are largeNguyễn Thái Ngọc Duy, Jul 5, 2016
  9. Junio C HamanoJul 12, 2016
  10. Junio C HamanoJul 12, 2016
  11. 2/5 sha1_file.c: use type off_t* for object_info->disk_sizepNguyễn Thái Ngọc Duy, Jul 5, 2016
  12. Junio C HamanoJul 12, 2016
  13. Junio C HamanoJul 12, 2016
  14. 3/5 index-pack: correct "len" type in unpack_data()Nguyễn Thái Ngọc Duy, Jul 5, 2016
  15. Johannes SixtJul 5, 2016
  16. Duy NguyenJul 6, 2016
  17. Junio C HamanoJul 6, 2016
  18. Junio C HamanoJul 6, 2016
  19. Christoph MichelbachJul 5, 2016
  20. Duy NguyenJul 10, 2016
  21. 6/5 pack-objects: do not truncate result in-pack object size on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 10, 2016
  22. 7/5 fsck: use streaming interface for large blobs in packNguyễn Thái Ngọc Duy, Jul 10, 2016
  23. Junio C HamanoJul 12, 2016
  24. Junio C HamanoJul 12, 2016
  25. Junio C HamanoJul 12, 2016
  26. Junio C HamanoJul 12, 2016
  27. Junio C HamanoJul 12, 2016
  28. Duy NguyenJul 13, 2016
  29. 0/7 Number truncation with 4+ GB files on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 13, 2016
  30. 4/7 index-pack: report correct bad object offsets even if they are largeNguyễn Thái Ngọc Duy, Jul 13, 2016
  31. 3/7 index-pack: correct "len" type in unpack_data()Nguyễn Thái Ngọc Duy, Jul 13, 2016
  32. 2/7 sha1_file.c: use type off_t* for object_info->disk_sizepNguyễn Thái Ngọc Duy, Jul 13, 2016
  33. 5/7 index-pack: correct "offset" type in unpack_entry_data()Nguyễn Thái Ngọc Duy, Jul 13, 2016
  34. 7/7 fsck: use streaming interface for large blobs in packNguyễn Thái Ngọc Duy, Jul 13, 2016
  35. 6/7 pack-objects: do not truncate result in-pack object size on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 13, 2016
  36. 1/7 pack-objects: pass length to check_pack_crc() without truncationNguyễn Thái Ngọc Duy, Jul 13, 2016
  37. Junio C HamanoJul 13, 2016
  38. Christoph MichelbachJul 6, 2016
  39. Duy NguyenJul 6, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.