git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 1/5] pack-objects: pass length to check_pack_crc() without truncation

From
Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Date
Jul 5, 2016, 17:05 UTC
Message-ID
<20160705170558.10906-2-pclouds@gmail.com>
In-Reply-To
<20160705170558.10906-1-pclouds@gmail.com>

On 32 bit systems with large file support, unsigned long is 32-bit while the two offsets in the subtraction expression (pack-objects has the exact same expression as in sha1_file.c but not shown in diff) are in 64-bit. If an in-pack object is larger than 2^32 len/datalen is truncated and we get a misleading "error: bad packed object CRC for ..." as a result.

Use off_t for len and datalen. check_pack_crc() already accepts this argument as off_t and can deal with 4+ GB.

Noticed-by: Christoph Michelbach <michelbach94@gmail.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
 builtin/pack-objects.c | 2 +-
 sha1_file.c            | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 8f5e358..df6ecd5 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -349,7 +349,7 @@ static unsigned long write_reuse_object(struct sha1file *f, struct object_entry
 	struct revindex_entry *revidx;
 	off_t offset;
 	enum object_type type = entry->type;
-	unsigned long datalen;
+	off_t datalen;
 	unsigned char header[10], dheader[10];
 	unsigned hdrlen;
 
diff --git a/sha1_file.c b/sha1_file.c
index d5e1121..cb571ac 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -2281,7 +2281,7 @@ void *unpack_entry(struct packed_git *p, off_t obj_offset,
 
 		if (do_check_packed_object_crc && p->index_version > 1) {
 			struct revindex_entry *revidx = find_pack_revindex(p, obj_offset);
-			unsigned long len = revidx[1].offset - obj_offset;
+			off_t len = revidx[1].offset - obj_offset;
 			if (check_pack_crc(p, &w_curs, obj_offset, len, revidx->nr)) {
 				const unsigned char *sha1 =
 					nth_packed_object_sha1(p, revidx->nr);
-- 
2.8.2.537.g0965dd9
Previous: Nguyễn Thái Ngọc DuyNext: Junio C Hamano
Message 5 of 39 in “[bug] Reliably Reproducible Bad Packing of Objects”
  1. Christoph MichelbachJun 24, 2016
  2. Duy NguyenJul 2, 2016
  3. Duy NguyenJul 2, 2016
  4. 0/5 Number truncation with 4+ GB files on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 5, 2016
  5. 1/5 pack-objects: pass length to check_pack_crc() without truncationNguyễn Thái Ngọc Duy, Jul 5, 2016
  6. Junio C HamanoJul 12, 2016
  7. 5/5 index-pack: correct "offset" type in unpack_entry_data()Nguyễn Thái Ngọc Duy, Jul 5, 2016
  8. 4/5 index-pack: report correct bad object offsets even if they are largeNguyễn Thái Ngọc Duy, Jul 5, 2016
  9. Junio C HamanoJul 12, 2016
  10. Junio C HamanoJul 12, 2016
  11. 2/5 sha1_file.c: use type off_t* for object_info->disk_sizepNguyễn Thái Ngọc Duy, Jul 5, 2016
  12. Junio C HamanoJul 12, 2016
  13. Junio C HamanoJul 12, 2016
  14. 3/5 index-pack: correct "len" type in unpack_data()Nguyễn Thái Ngọc Duy, Jul 5, 2016
  15. Johannes SixtJul 5, 2016
  16. Duy NguyenJul 6, 2016
  17. Junio C HamanoJul 6, 2016
  18. Junio C HamanoJul 6, 2016
  19. Christoph MichelbachJul 5, 2016
  20. Duy NguyenJul 10, 2016
  21. 6/5 pack-objects: do not truncate result in-pack object size on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 10, 2016
  22. 7/5 fsck: use streaming interface for large blobs in packNguyễn Thái Ngọc Duy, Jul 10, 2016
  23. Junio C HamanoJul 12, 2016
  24. Junio C HamanoJul 12, 2016
  25. Junio C HamanoJul 12, 2016
  26. Junio C HamanoJul 12, 2016
  27. Junio C HamanoJul 12, 2016
  28. Duy NguyenJul 13, 2016
  29. 0/7 Number truncation with 4+ GB files on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 13, 2016
  30. 4/7 index-pack: report correct bad object offsets even if they are largeNguyễn Thái Ngọc Duy, Jul 13, 2016
  31. 3/7 index-pack: correct "len" type in unpack_data()Nguyễn Thái Ngọc Duy, Jul 13, 2016
  32. 2/7 sha1_file.c: use type off_t* for object_info->disk_sizepNguyễn Thái Ngọc Duy, Jul 13, 2016
  33. 5/7 index-pack: correct "offset" type in unpack_entry_data()Nguyễn Thái Ngọc Duy, Jul 13, 2016
  34. 7/7 fsck: use streaming interface for large blobs in packNguyễn Thái Ngọc Duy, Jul 13, 2016
  35. 6/7 pack-objects: do not truncate result in-pack object size on 32-bit systemsNguyễn Thái Ngọc Duy, Jul 13, 2016
  36. 1/7 pack-objects: pass length to check_pack_crc() without truncationNguyễn Thái Ngọc Duy, Jul 13, 2016
  37. Junio C HamanoJul 13, 2016
  38. Christoph MichelbachJul 6, 2016
  39. Duy NguyenJul 6, 2016

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.