{"thread":{"id":"27598","subject":"[PATCH] unpack_sha1_file(): zlib can only process 4GB at a time","startedAt":"2011-06-09T20:14:52Z","lastAt":"2011-06-09T20:14:52Z","messageCount":1,"participants":["Junio C Hamano"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"169810","messageId":"7vr572wymb.fsf@alter.siamese.dyndns.org","threadId":"27598","inReplyTo":null,"subject":"[PATCH] unpack_sha1_file(): zlib can only process 4GB at a time","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-06-09T20:14:52Z","receivedAt":"2011-06-09T20:14:52Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"The same theme as \"unpack-objects\" patch.\n\nSigned-off-by: Junio C Hamano <gitster@pobox.com>\n---\n\n * On a beefy enough machine, you could register a large binary blob\n   and run fsck, fetch, or push to play with that object with these\n   patches, but there are many more places that aren't safe.\n\n sha1_file.c |   29 ++++++++++++++++++++---------\n 1 files changed, 20 insertions(+), 9 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 12a166f..ab9e962 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1257,7 +1257,7 @@ static int unpack_sha1_header(z_stream *stream, unsigned char *map, unsigned lon\n \t/* Get the data stream */\n \tmemset(stream, 0, sizeof(*stream));\n \tstream->next_in = map;\n-\tstream->avail_in = mapsize;\n+\tstream->avail_in = zlib_buf_cap(mapsize);\n \tstream->next_out = buffer;\n \tstream->avail_out = bufsiz;\n \n@@ -1291,11 +1291,12 @@ static int unpack_sha1_header(z_stream *stream, unsigned char *map, unsigned lon\n \treturn 0;\n }\n \n-static void *unpack_sha1_rest(z_stream *stream, void *buffer, unsigned long size, const unsigned char *sha1)\n+static void *unpack_sha1_rest(z_stream *stream, void *buffer, unsigned long mapsize,\n+\t\t\t      unsigned long size, const unsigned char *sha1)\n {\n-\tint bytes = strlen(buffer) + 1;\n+\tunsigned long bytes = strlen(buffer) + 1;\n \tunsigned char *buf = xmallocz(size);\n-\tunsigned long n;\n+\tunsigned long n, bytes_to_produce, bytes_to_consume;\n \tint status = Z_OK;\n \n \tn = stream->total_out - bytes;\n@@ -1303,6 +1304,8 @@ static void *unpack_sha1_rest(z_stream *stream, void *buffer, unsigned long size\n \t\tn = size;\n \tmemcpy(buf, (char *) buffer + bytes, n);\n \tbytes = n;\n+\n+\tbytes_to_consume = mapsize - stream->total_in;\n \tif (bytes <= size) {\n \t\t/*\n \t\t * The above condition must be (bytes <= size), not\n@@ -1318,17 +1321,25 @@ static void *unpack_sha1_rest(z_stream *stream, void *buffer, unsigned long size\n \t\t * went well with status == Z_STREAM_END at the end.\n \t\t */\n \t\tstream->next_out = buf + bytes;\n-\t\tstream->avail_out = size - bytes;\n-\t\twhile (status == Z_OK)\n+\t\tbytes_to_produce = size - bytes;\n+\n+\t\twhile (status == Z_OK || status == Z_BUF_ERROR) {\n+\t\t\tunsigned char *in0 = stream->next_in;\n+\t\t\tunsigned char *out0 = stream->next_out;\n+\t\t\tstream->avail_in = zlib_buf_cap(bytes_to_consume);\n+\t\t\tstream->avail_out = zlib_buf_cap(bytes_to_produce);\n \t\t\tstatus = git_inflate(stream, Z_FINISH);\n+\t\t\tbytes_to_produce -= stream->next_out - out0;\n+\t\t\tbytes_to_consume -= stream->next_in - in0;\n+\t\t}\n \t}\n-\tif (status == Z_STREAM_END && !stream->avail_in) {\n+\tif (status == Z_STREAM_END && !bytes_to_consume) {\n \t\tgit_inflate_end(stream);\n \t\treturn buf;\n \t}\n \n \tif (status < 0)\n-\t\terror(\"corrupt loose object '%s'\", sha1_to_hex(sha1));\n+\t\terror(\"corrupt loose object '%s' %d\", sha1_to_hex(sha1), status);\n \telse if (stream->avail_in)\n \t\terror(\"garbage at end of loose object '%s'\",\n \t\t      sha1_to_hex(sha1));\n@@ -1397,7 +1408,7 @@ static void *unpack_sha1_file(void *map, unsigned long mapsize, enum object_type\n \tif (ret < Z_OK || (*type = parse_sha1_header(hdr, size)) < 0)\n \t\treturn NULL;\n \n-\treturn unpack_sha1_rest(&stream, hdr, *size, sha1);\n+\treturn unpack_sha1_rest(&stream, hdr, mapsize, *size, sha1);\n }\n \n unsigned long get_size_from_delta(struct packed_git *p,\n-- \n1.7.6.rc1.118.ge175b4a\n"}]}