{"thread":{"id":"7592","subject":"[PATCH 12/10] validate reused pack data with CRC when possible","startedAt":"2007-04-10T04:15:41Z","lastAt":"2007-04-10T04:15:41Z","messageCount":1,"participants":["Nicolas Pitre"],"isPatch":true,"patchVersion":1,"patchTotal":10},"messages":[{"id":"38987","messageId":"alpine.LFD.0.98.0704100004200.28181@xanadu.home","threadId":"7592","inReplyTo":null,"subject":"[PATCH 12/10] validate reused pack data with CRC when possible","fromName":"Nicolas Pitre","fromEmail":"nico@cam.org","sentAt":"2007-04-10T04:15:41Z","receivedAt":"2007-04-10T04:15:41Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"This replaces the inflate validation with a CRC validation when reusing\ndata from a pack which uses index version 2.  That makes repacking much \nsafer against corruptions, and it should be a bit faster too.\n\nSigned-off-by: Nicolas Pitre <nico@cam.org>\n---\n\nThis completes the development for this patch series.  I tested that \nthis works as expected by manually corrupting a pack, etc.\nI intend to write real tests for this stuff of course, but at least the\nway should be completely clear for the pack spliting series now.\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex 099dea0..533dd99 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -233,12 +233,6 @@ static struct revindex_entry * find_packed_object(struct packed_git *p,\n \tdie(\"internal error: pack revindex corrupt\");\n }\n \n-static off_t find_packed_object_size(struct packed_git *p, off_t ofs)\n-{\n-\tstruct revindex_entry *entry = find_packed_object(p, ofs);\n-\treturn entry[1].offset - ofs;\n-}\n-\n static const unsigned char *find_packed_object_name(struct packed_git *p,\n \t\t\t\t\t\t    off_t ofs)\n {\n@@ -321,6 +315,28 @@ static int check_pack_inflate(struct packed_git *p,\n \t\tstream.total_in == len) ? 0 : -1;\n }\n \n+static int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,\n+\t\t\t  off_t offset, off_t len, unsigned int nr)\n+{\n+\tconst uint32_t *index_crc;\n+\tuint32_t data_crc = crc32(0, Z_NULL, 0);\n+\n+\tdo {\n+\t\tunsigned int avail;\n+\t\tvoid *data = use_pack(p, w_curs, offset, &avail);\n+\t\tif (avail > len)\n+\t\t\tavail = len;\n+\t\tdata_crc = crc32(data_crc, data, avail);\n+\t\toffset += avail;\n+\t\tlen -= avail;\n+\t} while (len);\n+\n+\tindex_crc = p->index_data;\n+\tindex_crc += 2 + 256 + p->num_objects * (20/4) + nr;\n+\n+\treturn data_crc != ntohl(*index_crc);\n+}\n+\n static void copy_pack_data(struct sha1file *f,\n \t\tstruct packed_git *p,\n \t\tstruct pack_window **w_curs,\n@@ -485,6 +501,7 @@ static unsigned long write_object(struct sha1file *f,\n \telse {\n \t\tstruct packed_git *p = entry->in_pack;\n \t\tstruct pack_window *w_curs = NULL;\n+\t\tstruct revindex_entry *revidx;\n \t\toff_t offset;\n \n \t\tif (entry->delta) {\n@@ -507,12 +524,17 @@ static unsigned long write_object(struct sha1file *f,\n \t\t\thdrlen += 20;\n \t\t}\n \n-\t\toffset = entry->in_pack_offset + entry->in_pack_header_size;\n-\t\tdatalen = find_packed_object_size(p, entry->in_pack_offset)\n-\t\t\t\t- entry->in_pack_header_size;\n-\t\tif (!pack_to_stdout && check_pack_inflate(p, &w_curs,\n-\t\t\t\toffset, datalen, entry->size))\n-\t\t\tdie(\"corrupt delta in pack %s\", sha1_to_hex(entry->sha1));\n+\t\toffset = entry->in_pack_offset;\n+\t\trevidx = find_packed_object(p, offset);\n+\t\tdatalen = revidx[1].offset - offset;\n+\t\tif (!pack_to_stdout && p->index_version > 1 &&\n+\t\t    check_pack_crc(p, &w_curs, offset, datalen, revidx->nr))\n+\t\t\tdie(\"bad packed object CRC for %s\", sha1_to_hex(entry->sha1));\n+\t\toffset += entry->in_pack_header_size;\n+\t\tdatalen -= entry->in_pack_header_size;\n+\t\tif (!pack_to_stdout && p->index_version == 1 &&\n+\t\t    check_pack_inflate(p, &w_curs, offset, datalen, entry->size))\n+\t\t\tdie(\"corrupt packed object for %s\", sha1_to_hex(entry->sha1));\n \t\tcopy_pack_data(f, p, &w_curs, offset, datalen);\n \t\tunuse_pack(&w_curs);\n \t\treused++;\n"}]}