{"thread":{"id":"33192","subject":"[PATCH] archive-zip: use deflateInit2() to ask for raw compressed data","startedAt":"2013-03-15T22:21:51Z","lastAt":"2013-03-15T22:21:51Z","messageCount":1,"participants":["René Scharfe"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"211421","messageId":"51439EFF.3000607@lsrfire.ath.cx","threadId":"33192","inReplyTo":null,"subject":"[PATCH] archive-zip: use deflateInit2() to ask for raw compressed data","fromName":"René Scharfe","fromEmail":"rene.scharfe@lsrfire.ath.cx","sentAt":"2013-03-15T22:21:51Z","receivedAt":"2013-03-15T22:21:51Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"We use the function git_deflate_init() -- which wraps the zlib function\ndeflateInit() -- to initialize compression of ZIP file entries.  This\nresults in compressed data prefixed with a two-bytes long header and\nfollowed by a four-bytes trailer.  ZIP file entries consist of ZIP\nheaders and raw compressed data instead, so we remove the zlib wrapper\nbefore writing the result.\n\nWe can ask zlib for the the raw compressed data without the unwanted\nparts in the first place by using deflateInit2() and specifying a\nnegative number of bits to size the window.  For that purpose, factor\nout the function do_git_deflate_init() and add git_deflate_init_raw(),\nwhich wraps it.  Then use the latter in archive-zip.c and get rid of\nthe code that stripped the zlib header and trailer.\n\nAlso rename the helper function zlib_deflate() to zlib_deflate_raw()\nto reflect the change.\n\nThus we avoid generating data that we throw away anyway, the code\nbecomes shorter and some magic constants are removed.\n\nSigned-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>\n---\n archive-zip.c | 36 ++++++++++++++----------------------\n cache.h       |  1 +\n zlib.c        | 25 +++++++++++++++++++------\n 3 files changed, 34 insertions(+), 28 deletions(-)\n\ndiff --git a/archive-zip.c b/archive-zip.c\nindex d3aef53..e744d77 100644\n--- a/archive-zip.c\n+++ b/archive-zip.c\n@@ -111,8 +111,9 @@ static void copy_le32(unsigned char *dest, unsigned int n)\n \tdest[3] = 0xff & (n >> 030);\n }\n \n-static void *zlib_deflate(void *data, unsigned long size,\n-\t\tint compression_level, unsigned long *compressed_size)\n+static void *zlib_deflate_raw(void *data, unsigned long size,\n+\t\t\t      int compression_level,\n+\t\t\t      unsigned long *compressed_size)\n {\n \tgit_zstream stream;\n \tunsigned long maxsize;\n@@ -120,7 +121,7 @@ static void *zlib_deflate(void *data, unsigned long size,\n \tint result;\n \n \tmemset(&stream, 0, sizeof(stream));\n-\tgit_deflate_init(&stream, compression_level);\n+\tgit_deflate_init_raw(&stream, compression_level);\n \tmaxsize = git_deflate_bound(&stream, size);\n \tbuffer = xmalloc(maxsize);\n \n@@ -265,14 +266,11 @@ static int write_zip_entry(struct archiver_args *args,\n \t}\n \n \tif (buffer && method == 8) {\n-\t\tdeflated = zlib_deflate(buffer, size, args->compression_level,\n-\t\t\t\t&compressed_size);\n-\t\tif (deflated && compressed_size - 6 < size) {\n-\t\t\t/* ZLIB --> raw compressed data (see RFC 1950) */\n-\t\t\t/* CMF and FLG ... */\n-\t\t\tout = (unsigned char *)deflated + 2;\n-\t\t\tcompressed_size -= 6;\t/* ... and ADLER32 */\n-\t\t} else {\n+\t\tout = deflated = zlib_deflate_raw(buffer, size,\n+\t\t\t\t\t\t  args->compression_level,\n+\t\t\t\t\t\t  &compressed_size);\n+\t\tif (!out || compressed_size >= size) {\n+\t\t\tout = buffer;\n \t\t\tmethod = 0;\n \t\t\tcompressed_size = size;\n \t\t}\n@@ -353,7 +351,7 @@ static int write_zip_entry(struct archiver_args *args,\n \t\tunsigned char compressed[STREAM_BUFFER_SIZE * 2];\n \n \t\tmemset(&zstream, 0, sizeof(zstream));\n-\t\tgit_deflate_init(&zstream, args->compression_level);\n+\t\tgit_deflate_init_raw(&zstream, args->compression_level);\n \n \t\tcompressed_size = 0;\n \t\tzstream.next_out = compressed;\n@@ -370,13 +368,10 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\tresult = git_deflate(&zstream, 0);\n \t\t\tif (result != Z_OK)\n \t\t\t\tdie(\"deflate error (%d)\", result);\n-\t\t\tout = compressed;\n-\t\t\tif (!compressed_size)\n-\t\t\t\tout += 2;\n-\t\t\tout_len = zstream.next_out - out;\n+\t\t\tout_len = zstream.next_out - compressed;\n \n \t\t\tif (out_len > 0) {\n-\t\t\t\twrite_or_die(1, out, out_len);\n+\t\t\t\twrite_or_die(1, compressed, out_len);\n \t\t\t\tcompressed_size += out_len;\n \t\t\t\tzstream.next_out = compressed;\n \t\t\t\tzstream.avail_out = sizeof(compressed);\n@@ -394,11 +389,8 @@ static int write_zip_entry(struct archiver_args *args,\n \t\t\tdie(\"deflate error (%d)\", result);\n \n \t\tgit_deflate_end(&zstream);\n-\t\tout = compressed;\n-\t\tif (!compressed_size)\n-\t\t\tout += 2;\n-\t\tout_len = zstream.next_out - out - 4;\n-\t\twrite_or_die(1, out, out_len);\n+\t\tout_len = zstream.next_out - compressed;\n+\t\twrite_or_die(1, compressed, out_len);\n \t\tcompressed_size += out_len;\n \t\tzip_offset += compressed_size;\n \ndiff --git a/cache.h b/cache.h\nindex e493563..81a39a2 100644\n--- a/cache.h\n+++ b/cache.h\n@@ -34,6 +34,7 @@ int git_inflate(git_zstream *, int flush);\n \n void git_deflate_init(git_zstream *, int level);\n void git_deflate_init_gzip(git_zstream *, int level);\n+void git_deflate_init_raw(git_zstream *, int level);\n void git_deflate_end(git_zstream *);\n int git_deflate_abort(git_zstream *);\n int git_deflate_end_gently(git_zstream *);\ndiff --git a/zlib.c b/zlib.c\nindex 2b2c0c7..bbaa081 100644\n--- a/zlib.c\n+++ b/zlib.c\n@@ -168,13 +168,8 @@ void git_deflate_init(git_zstream *strm, int level)\n \t    strm->z.msg ? strm->z.msg : \"no message\");\n }\n \n-void git_deflate_init_gzip(git_zstream *strm, int level)\n+static void do_git_deflate_init(git_zstream *strm, int level, int windowBits)\n {\n-\t/*\n-\t * Use default 15 bits, +16 is to generate gzip header/trailer\n-\t * instead of the zlib wrapper.\n-\t */\n-\tconst int windowBits = 15 + 16;\n \tint status;\n \n \tzlib_pre_call(strm);\n@@ -188,6 +183,24 @@ void git_deflate_init_gzip(git_zstream *strm, int level)\n \t    strm->z.msg ? strm->z.msg : \"no message\");\n }\n \n+void git_deflate_init_gzip(git_zstream *strm, int level)\n+{\n+\t/*\n+\t * Use default 15 bits, +16 is to generate gzip header/trailer\n+\t * instead of the zlib wrapper.\n+\t */\n+\treturn do_git_deflate_init(strm, level, 15 + 16);\n+}\n+\n+void git_deflate_init_raw(git_zstream *strm, int level)\n+{\n+\t/*\n+\t * Use default 15 bits, negate the value to get raw compressed\n+\t * data without zlib header and trailer.\n+\t */\n+\treturn do_git_deflate_init(strm, level, -15);\n+}\n+\n int git_deflate_abort(git_zstream *strm)\n {\n \tint status;\n-- \n1.8.2\n"}]}