{"thread":{"id":"23436","subject":"[PATCH] index-pack: smarter memory usage when appending objects","startedAt":"2010-04-12T20:50:35Z","lastAt":"2010-04-12T20:50:35Z","messageCount":1,"participants":["Nicolas Pitre"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"139374","messageId":"alpine.LFD.2.00.1004121639170.7232@xanadu.home","threadId":"23436","inReplyTo":null,"subject":"[PATCH] index-pack: smarter memory usage when appending objects","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-04-12T20:50:35Z","receivedAt":"2010-04-12T20:50:35Z","isPatch":true,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"In the same spirit as commit 9892bebafe, let's avoid allocating the full\nbuffer for the deflated data in write_compressed() in order to write it.\nLet's deflate and write the data in chunks instead to reduce memory\nusage.\n\nSigned-off-by: Nicolas Pitre <nico@fluxnic.net>\n\ndiff --git a/builtin/index-pack.c b/builtin/index-pack.c\nindex 4308abb..c1e0394 100644\n--- a/builtin/index-pack.c\n+++ b/builtin/index-pack.c\n@@ -669,25 +669,25 @@ static void parse_pack_objects(unsigned char *sha1)\n static int write_compressed(struct sha1file *f, void *in, unsigned int size)\n {\n \tz_stream stream;\n-\tunsigned long maxsize;\n-\tvoid *out;\n+\tint status;\n+\tunsigned char outbuf[4096];\n \n \tmemset(&stream, 0, sizeof(stream));\n \tdeflateInit(&stream, zlib_compression_level);\n-\tmaxsize = deflateBound(&stream, size);\n-\tout = xmalloc(maxsize);\n-\n-\t/* Compress it */\n \tstream.next_in = in;\n \tstream.avail_in = size;\n-\tstream.next_out = out;\n-\tstream.avail_out = maxsize;\n-\twhile (deflate(&stream, Z_FINISH) == Z_OK);\n-\tdeflateEnd(&stream);\n \n+\tdo {\n+\t\tstream.next_out = outbuf;\n+\t\tstream.avail_out = sizeof(outbuf);\n+\t\tstatus = deflate(&stream, Z_FINISH);\n+\t\tsha1write(f, outbuf, sizeof(outbuf) - stream.avail_out);\n+\t} while (status == Z_OK); \n+\n+\tif (status != Z_STREAM_END)\n+\t\tdie(\"unable to deflate appended object (%d)\", status);\n \tsize = stream.total_out;\n-\tsha1write(f, out, size);\n-\tfree(out);\n+\tdeflateEnd(&stream);\n \treturn size;\n }\n \n"}]}