{"thread":{"id":"11839","subject":"[PATCH 01/11] Introduce stream compress helpers","startedAt":"2008-02-02T11:35:46Z","lastAt":"2008-02-04T02:08:02Z","messageCount":21,"participants":["Marco Costalba","Junio C Hamano","Shawn O. Pearce"],"isPatch":true,"patchVersion":1,"patchTotal":11},"messages":[{"id":"67119","messageId":"1201952156-6764-1-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":null,"subject":"[PATCH 01/11] Introduce stream compress helpers","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:46Z","receivedAt":"2008-02-02T11:35:46Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"When decompressing a zlib stream use this\nhelpers instead of calling low level zlib\nfunction.\n\nThis patch introduces the necessary framework,\nstill no code change.\n\nThis is the first step in generalizing compress and\ndecompress functions avoiding zlib directly calls.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n Makefile   |    4 ++--\n compress.c |   58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n compress.h |   12 ++++++++++++\n 3 files changed, 72 insertions(+), 2 deletions(-)\n create mode 100644 compress.c\n create mode 100644 compress.h\n\ndiff --git a/Makefile b/Makefile\nindex 91a460f..f70f995 100644\n--- a/Makefile\n+++ b/Makefile\n@@ -292,7 +292,7 @@ LIB_FILE=libgit.a\n XDIFF_LIB=xdiff/lib.a\n \n LIB_H = \\\n-\tarchive.h blob.h cache.h cache-tree.h commit.h csum-file.h delta.h grep.h \\\n+\tarchive.h blob.h cache.h cache-tree.h commit.h compress.h csum-file.h delta.h grep.h \\\n \tdiff.h object.h pack.h pkt-line.h quote.h refs.h list-objects.h sideband.h \\\n \trun-command.h strbuf.h tag.h tree.h git-compat-util.h revision.h \\\n \ttree-walk.h log-tree.h dir.h path-list.h unpack-trees.h builtin.h \\\n@@ -305,7 +305,7 @@ DIFF_OBJS = \\\n \tdiffcore-delta.o log-tree.o\n \n LIB_OBJS = \\\n-\tblob.o commit.o connect.o csum-file.o cache-tree.o base85.o \\\n+\tblob.o commit.o compress.o connect.o csum-file.o cache-tree.o base85.o \\\n \tdate.o diff-delta.o entry.o exec_cmd.o ident.o \\\n \tpretty.o interpolate.o hash.o \\\n \tlockfile.o \\\ndiff --git a/compress.c b/compress.c\nnew file mode 100644\nindex 0000000..f6986c3\n--- /dev/null\n+++ b/compress.c\n@@ -0,0 +1,58 @@\n+#include \"cache.h\"\n+#include \"compress.h\"\n+\n+unsigned long compress_alloc(z_stream *stream, int level, unsigned long size)\n+{\n+\tmemset(stream, 0, sizeof(*stream));\n+\tdeflateInit(stream, level);\n+\treturn deflateBound(stream, size);\n+}\n+\n+int compress_start(z_stream *stream,\n+                   unsigned char *in, unsigned long in_size,\n+                   unsigned char *out, unsigned long out_size)\n+{\n+\tstream->next_out = (out ? out : xmalloc(out_size));\n+\tstream->avail_out = out_size;\n+\tstream->next_in = in;\n+\tstream->avail_in = in_size;\n+\treturn Z_OK;\n+}\n+\n+int compress_next(z_stream *stream, int flush)\n+{\n+\tint result;\n+\n+\tdo {\n+\t\tresult = deflate(stream, flush);\n+\t} while (result == Z_OK);\n+\n+\treturn result;\n+}\n+\n+unsigned long compress_free(z_stream *stream)\n+{\n+\tdeflateEnd(stream);\n+\treturn stream->total_out;\n+}\n+\n+unsigned long compress_all(int level, unsigned char *data,\n+                           unsigned long size, unsigned char **out)\n+{\n+\tint bound, result;\n+\tz_stream stream;\n+\n+\tbound = compress_alloc(&stream, level, size);\n+\tcompress_start(&stream, data, size, NULL, bound);\n+\n+\t*out = stream.next_out;\n+\tresult = compress_next(&stream, Z_FINISH);\n+\n+\tif (result != Z_STREAM_END) {\n+\t\tcompress_free(&stream);\n+\t\tfree(*out);\n+\t\t*out = NULL;\n+\t\treturn 0;\n+\t}\n+\treturn compress_free(&stream);\n+}\ndiff --git a/compress.h b/compress.h\nnew file mode 100644\nindex 0000000..d73c365\n--- /dev/null\n+++ b/compress.h\n@@ -0,0 +1,12 @@\n+#ifndef COMPRESS_H\n+#define COMPRESS_H\n+\n+extern unsigned long compress_alloc(z_stream *stream, int level, unsigned long size);\n+extern int compress_start(z_stream *stream, unsigned char *in, unsigned long in_size,\n+                           unsigned char *out, unsigned long out_size);\n+extern int compress_next(z_stream *stream, int flush);\n+extern unsigned long compress_free(z_stream *stream);\n+extern unsigned long compress_all(int level, unsigned char *data, unsigned long size,\n+                                  unsigned char **out);\n+\n+#endif\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67120","messageId":"1201952156-6764-2-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-1-git-send-email-mcostalba@gmail.com","subject":"[PATCH 02/11] Use new compress helpers in git files","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:47Z","receivedAt":"2008-02-02T11:35:47Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"These are the 'easy' ones, where a signgle step\ncompression is requested so that we can use only\none call to compress_all()\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n archive-zip.c          |   28 +++-------------------------\n builtin-pack-objects.c |   19 +++----------------\n diff.c                 |   22 +++++-----------------\n index-pack.c           |   20 +++-----------------\n 4 files changed, 14 insertions(+), 75 deletions(-)\n\ndiff --git a/archive-zip.c b/archive-zip.c\nindex 74e30f6..9071b86 100644\n--- a/archive-zip.c\n+++ b/archive-zip.c\n@@ -3,6 +3,7 @@\n  */\n #include \"cache.h\"\n #include \"commit.h\"\n+#include \"compress.h\"\n #include \"blob.h\"\n #include \"tree.h\"\n #include \"quote.h\"\n@@ -97,33 +98,10 @@ static void copy_le32(unsigned char *dest, unsigned int n)\n static void *zlib_deflate(void *data, unsigned long size,\n                           unsigned long *compressed_size)\n {\n-\tz_stream stream;\n-\tunsigned long maxsize;\n-\tvoid *buffer;\n-\tint result;\n-\n-\tmemset(&stream, 0, sizeof(stream));\n-\tdeflateInit(&stream, zlib_compression_level);\n-\tmaxsize = deflateBound(&stream, size);\n-\tbuffer = xmalloc(maxsize);\n-\n-\tstream.next_in = data;\n-\tstream.avail_in = size;\n-\tstream.next_out = buffer;\n-\tstream.avail_out = maxsize;\n-\n-\tdo {\n-\t\tresult = deflate(&stream, Z_FINISH);\n-\t} while (result == Z_OK);\n-\n-\tif (result != Z_STREAM_END) {\n-\t\tfree(buffer);\n-\t\treturn NULL;\n-\t}\n \n-\tdeflateEnd(&stream);\n-\t*compressed_size = stream.total_out;\n+\tunsigned char *buffer = NULL;\n \n+\t*compressed_size = compress_all(zlib_compression_level, data, size, &buffer);\n \treturn buffer;\n }\n \ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex d3efeff..991a30f 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"compress.h\"\n #include \"attr.h\"\n #include \"object.h\"\n #include \"blob.h\"\n@@ -409,9 +410,7 @@ static unsigned long write_object(struct sha1file *f,\n \t\t\t\t */\n \n \tif (!to_reuse) {\n-\t\tz_stream stream;\n-\t\tunsigned long maxsize;\n-\t\tvoid *out;\n+\t\tunsigned char *out = NULL;\n \t\tif (!usable_delta) {\n \t\t\tbuf = read_sha1_file(entry->idx.sha1, &obj_type, &size);\n \t\t\tif (!buf)\n@@ -432,19 +431,7 @@ static unsigned long write_object(struct sha1file *f,\n \t\t\t\tOBJ_OFS_DELTA : OBJ_REF_DELTA;\n \t\t}\n \t\t/* compress the data to store and put compressed length in datalen */\n-\t\tmemset(&stream, 0, sizeof(stream));\n-\t\tdeflateInit(&stream, pack_compression_level);\n-\t\tmaxsize = deflateBound(&stream, size);\n-\t\tout = xmalloc(maxsize);\n-\t\t/* Compress it */\n-\t\tstream.next_in = buf;\n-\t\tstream.avail_in = size;\n-\t\tstream.next_out = out;\n-\t\tstream.avail_out = maxsize;\n-\t\twhile (deflate(&stream, Z_FINISH) == Z_OK)\n-\t\t\t/* nothing */;\n-\t\tdeflateEnd(&stream);\n-\t\tdatalen = stream.total_out;\n+\t\tdatalen = compress_all(pack_compression_level, buf, size, &out);\n \n \t\t/*\n \t\t * The object header is a byte of 'type' followed by zero or\ndiff --git a/diff.c b/diff.c\nindex d464fe3..9eb9672 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -2,6 +2,7 @@\n  * Copyright (C) 2005 Junio C Hamano\n  */\n #include \"cache.h\"\n+#include \"compress.h\"\n #include \"quote.h\"\n #include \"diff.h\"\n #include \"diffcore.h\"\n@@ -1037,23 +1038,10 @@ static unsigned char *deflate_it(char *data,\n \t\t\t\t unsigned long size,\n \t\t\t\t unsigned long *result_size)\n {\n-\tint bound;\n-\tunsigned char *deflated;\n-\tz_stream stream;\n-\n-\tmemset(&stream, 0, sizeof(stream));\n-\tdeflateInit(&stream, zlib_compression_level);\n-\tbound = deflateBound(&stream, size);\n-\tdeflated = xmalloc(bound);\n-\tstream.next_out = deflated;\n-\tstream.avail_out = bound;\n-\n-\tstream.next_in = (unsigned char *)data;\n-\tstream.avail_in = size;\n-\twhile (deflate(&stream, Z_FINISH) == Z_OK)\n-\t\t; /* nothing */\n-\tdeflateEnd(&stream);\n-\t*result_size = stream.total_out;\n+\tunsigned char *deflated = NULL;\n+\n+\t*result_size = compress_all(zlib_compression_level,\n+                                   (unsigned char *)data, size, &deflated);\n \treturn deflated;\n }\n \ndiff --git a/index-pack.c b/index-pack.c\nindex 9fd6982..880088e 100644\n--- a/index-pack.c\n+++ b/index-pack.c\n@@ -1,4 +1,5 @@\n #include \"cache.h\"\n+#include \"compress.h\"\n #include \"delta.h\"\n #include \"pack.h\"\n #include \"csum-file.h\"\n@@ -494,24 +495,9 @@ static void parse_pack_objects(unsigned char *sha1)\n \n static int write_compressed(int fd, void *in, unsigned int size, uint32_t *obj_crc)\n {\n-\tz_stream stream;\n-\tunsigned long maxsize;\n-\tvoid *out;\n+\tunsigned char *out = NULL;\n \n-\tmemset(&stream, 0, sizeof(stream));\n-\tdeflateInit(&stream, zlib_compression_level);\n-\tmaxsize = deflateBound(&stream, size);\n-\tout = xmalloc(maxsize);\n-\n-\t/* Compress it */\n-\tstream.next_in = in;\n-\tstream.avail_in = size;\n-\tstream.next_out = out;\n-\tstream.avail_out = maxsize;\n-\twhile (deflate(&stream, Z_FINISH) == Z_OK);\n-\tdeflateEnd(&stream);\n-\n-\tsize = stream.total_out;\n+\tsize = compress_all(zlib_compression_level, in, size, &out);\n \twrite_or_die(fd, out, size);\n \t*obj_crc = crc32(*obj_crc, out, size);\n \tfree(out);\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67121","messageId":"1201952156-6764-3-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-2-git-send-email-mcostalba@gmail.com","subject":"[PATCH 03/11] Use new compress helpers in fast-import","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:48Z","receivedAt":"2008-02-02T11:35:48Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Here is slightly more difficult, in particular\na xrealloc() has been substituted with a\nfree() + xmalloc() to keep the code simple.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n fast-import.c |   45 +++++++++++++++------------------------------\n 1 files changed, 15 insertions(+), 30 deletions(-)\n\ndiff --git a/fast-import.c b/fast-import.c\nindex a523b17..b6bb84c 100644\n--- a/fast-import.c\n+++ b/fast-import.c\n@@ -141,6 +141,7 @@ Format of STDIN stream:\n \n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"compress.h\"\n #include \"object.h\"\n #include \"blob.h\"\n #include \"tree.h\"\n@@ -997,13 +998,13 @@ static int store_object(\n \tunsigned char *sha1out,\n \tuintmax_t mark)\n {\n-\tvoid *out, *delta;\n+\tunsigned char *out, *delta;\n \tstruct object_entry *e;\n \tunsigned char hdr[96];\n \tunsigned char sha1[20];\n \tunsigned long hdrlen, deltalen;\n \tSHA_CTX c;\n-\tz_stream s;\n+\tint out_size;\n \n \thdrlen = sprintf((char*)hdr,\"%s %lu\", typename(type),\n \t\t(unsigned long)dat->len) + 1;\n@@ -1039,24 +1040,15 @@ static int store_object(\n \t} else\n \t\tdelta = NULL;\n \n-\tmemset(&s, 0, sizeof(s));\n-\tdeflateInit(&s, pack_compression_level);\n-\tif (delta) {\n-\t\ts.next_in = delta;\n-\t\ts.avail_in = deltalen;\n-\t} else {\n-\t\ts.next_in = (void *)dat->buf;\n-\t\ts.avail_in = dat->len;\n-\t}\n-\ts.avail_out = deflateBound(&s, s.avail_in);\n-\ts.next_out = out = xmalloc(s.avail_out);\n-\twhile (deflate(&s, Z_FINISH) == Z_OK)\n-\t\t/* nothing */;\n-\tdeflateEnd(&s);\n+\tif (delta)\n+\t\tout_size = compress_all(pack_compression_level, delta, deltalen, &out);\n+\telse\n+\t\tout_size = compress_all(pack_compression_level,\n+                                       (unsigned char *)dat->buf, dat->len, &out);\n \n \t/* Determine if we should auto-checkpoint. */\n-\tif ((pack_size + 60 + s.total_out) > max_packsize\n-\t\t|| (pack_size + 60 + s.total_out) < pack_size) {\n+\tif ((pack_size + 60 + out_size) > max_packsize\n+\t\t|| (pack_size + 60 + out_size) < pack_size) {\n \n \t\t/* This new object needs to *not* have the current pack_id. */\n \t\te->pack_id = pack_id + 1;\n@@ -1066,16 +1058,9 @@ static int store_object(\n \t\tif (delta) {\n \t\t\tfree(delta);\n \t\t\tdelta = NULL;\n-\n-\t\t\tmemset(&s, 0, sizeof(s));\n-\t\t\tdeflateInit(&s, pack_compression_level);\n-\t\t\ts.next_in = (void *)dat->buf;\n-\t\t\ts.avail_in = dat->len;\n-\t\t\ts.avail_out = deflateBound(&s, s.avail_in);\n-\t\t\ts.next_out = out = xrealloc(out, s.avail_out);\n-\t\t\twhile (deflate(&s, Z_FINISH) == Z_OK)\n-\t\t\t\t/* nothing */;\n-\t\t\tdeflateEnd(&s);\n+\t\t\tfree(out);\n+\t\t\tout_size = compress_all(pack_compression_level,\n+                                               (unsigned char *)dat->buf, dat->len, &out);\n \t\t}\n \t}\n \n@@ -1108,8 +1093,8 @@ static int store_object(\n \t\tpack_size += hdrlen;\n \t}\n \n-\twrite_or_die(pack_data->pack_fd, out, s.total_out);\n-\tpack_size += s.total_out;\n+\twrite_or_die(pack_data->pack_fd, out, out_size);\n+\tpack_size += out_size;\n \n \tfree(out);\n \tfree(delta);\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67122","messageId":"1201952156-6764-4-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-3-git-send-email-mcostalba@gmail.com","subject":"[PATCH 04/11] Use new compress helpers in http-push.c","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:49Z","receivedAt":"2008-02-02T11:35:49Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"A multistep compress is required here, so\nwe need the full arsenal of compress helpers.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n http-push.c |   22 ++++++++--------------\n 1 files changed, 8 insertions(+), 14 deletions(-)\n\ndiff --git a/http-push.c b/http-push.c\nindex b2b410d..a7997ec 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"commit.h\"\n+#include \"compress.h\"\n #include \"pack.h\"\n #include \"tag.h\"\n #include \"blob.h\"\n@@ -491,31 +492,24 @@ static void start_put(struct transfer_request *request)\n \thdrlen = sprintf(hdr, \"%s %lu\", typename(type), len) + 1;\n \n \t/* Set it up */\n-\tmemset(&stream, 0, sizeof(stream));\n-\tdeflateInit(&stream, zlib_compression_level);\n-\tsize = deflateBound(&stream, len + hdrlen);\n+\tsize = compress_alloc(&stream, zlib_compression_level, len + hdrlen);\n \tstrbuf_init(&request->buffer.buf, size);\n \trequest->buffer.posn = 0;\n \n \t/* Compress it */\n-\tstream.next_out = (unsigned char *)request->buffer.buf.buf;\n-\tstream.avail_out = size;\n+\tcompress_start(&stream, (void *)hdr, hdrlen,\n+                      (unsigned char *)request->buffer.buf.buf, size);\n \n \t/* First header.. */\n-\tstream.next_in = (void *)hdr;\n-\tstream.avail_in = hdrlen;\n-\twhile (deflate(&stream, 0) == Z_OK)\n-\t\t/* nothing */;\n+\tcompress_next(&stream, Z_NO_FLUSH);\n \n \t/* Then the data itself.. */\n \tstream.next_in = unpacked;\n \tstream.avail_in = len;\n-\twhile (deflate(&stream, Z_FINISH) == Z_OK)\n-\t\t/* nothing */;\n-\tdeflateEnd(&stream);\n-\tfree(unpacked);\n+\tcompress_next(&stream, Z_FINISH);\n \n-\trequest->buffer.buf.len = stream.total_out;\n+\trequest->buffer.buf.len = compress_free(&stream);\n+\tfree(unpacked);\n \n \trequest->url = xmalloc(strlen(remote->url) +\n \t\t\t       strlen(request->lock->token) + 51);\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67123","messageId":"1201952156-6764-5-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-4-git-send-email-mcostalba@gmail.com","subject":"[PATCH 05/11] Use new compress helpers in sha1_file.c","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:50Z","receivedAt":"2008-02-02T11:35:50Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"A multistep compress is required here, so\nwe need the full arsenal of compress helpers.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n sha1_file.c |   41 ++++++++++++-----------------------------\n 1 files changed, 12 insertions(+), 29 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 66a4e00..f48ad04 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -7,6 +7,7 @@\n  * creation etc.\n  */\n #include \"cache.h\"\n+#include \"compress.h\"\n #include \"delta.h\"\n #include \"pack.h\"\n #include \"blob.h\"\n@@ -2102,33 +2103,23 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha\n \t}\n \n \t/* Set it up */\n-\tmemset(&stream, 0, sizeof(stream));\n-\tdeflateInit(&stream, zlib_compression_level);\n-\tsize = 8 + deflateBound(&stream, len+hdrlen);\n+\tsize = 8 + compress_alloc(&stream, zlib_compression_level, len+hdrlen);\n \tcompressed = xmalloc(size);\n \n \t/* Compress it */\n-\tstream.next_out = compressed;\n-\tstream.avail_out = size;\n+\tcompress_start(&stream, (unsigned char *)hdr, hdrlen, compressed, size);\n \n \t/* First header.. */\n-\tstream.next_in = (unsigned char *)hdr;\n-\tstream.avail_in = hdrlen;\n-\twhile (deflate(&stream, 0) == Z_OK)\n-\t\t/* nothing */;\n+\tcompress_next(&stream, Z_NO_FLUSH);\n \n \t/* Then the data itself.. */\n \tstream.next_in = buf;\n \tstream.avail_in = len;\n-\tret = deflate(&stream, Z_FINISH);\n+\tret = compress_next(&stream, Z_FINISH);\n \tif (ret != Z_STREAM_END)\n \t\tdie(\"unable to deflate new object %s (%d)\", sha1_to_hex(sha1), ret);\n \n-\tret = deflateEnd(&stream);\n-\tif (ret != Z_OK)\n-\t\tdie(\"deflateEnd on object %s failed (%d)\", sha1_to_hex(sha1), ret);\n-\n-\tsize = stream.total_out;\n+\tsize = compress_free(&stream);\n \n \tif (write_buffer(fd, compressed, size) < 0)\n \t\tdie(\"unable to write sha1 file\");\n@@ -2163,30 +2154,22 @@ static void *repack_object(const unsigned char *sha1, unsigned long *objsize)\n \thdrlen = sprintf(hdr, \"%s %lu\", typename(type), len) + 1;\n \n \t/* Set it up */\n-\tmemset(&stream, 0, sizeof(stream));\n-\tdeflateInit(&stream, zlib_compression_level);\n-\tsize = deflateBound(&stream, len + hdrlen);\n+\tsize = compress_alloc(&stream, zlib_compression_level, len + hdrlen);\n \tbuf = xmalloc(size);\n \n \t/* Compress it */\n-\tstream.next_out = buf;\n-\tstream.avail_out = size;\n+\tcompress_start(&stream, (unsigned char *)hdr, hdrlen, buf, size);\n \n \t/* First header.. */\n-\tstream.next_in = (void *)hdr;\n-\tstream.avail_in = hdrlen;\n-\twhile (deflate(&stream, 0) == Z_OK)\n-\t\t/* nothing */;\n+\tcompress_next(&stream, Z_NO_FLUSH);\n \n \t/* Then the data itself.. */\n \tstream.next_in = unpacked;\n \tstream.avail_in = len;\n-\twhile (deflate(&stream, Z_FINISH) == Z_OK)\n-\t\t/* nothing */;\n-\tdeflateEnd(&stream);\n-\tfree(unpacked);\n+\tcompress_next(&stream, Z_FINISH);\n \n-\t*objsize = stream.total_out;\n+\t*objsize = compress_free(&stream);\n+\tfree(unpacked);\n \treturn buf;\n }\n \n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67127","messageId":"1201952156-6764-6-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-5-git-send-email-mcostalba@gmail.com","subject":"[PATCH 06/11] Better error handling in compress_all()","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:51Z","receivedAt":"2008-02-02T11:35:51Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Also let the caller to xmalloc() the buffer\nint compress_start()\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n compress.c |   19 +++++++++----------\n 1 files changed, 9 insertions(+), 10 deletions(-)\n\ndiff --git a/compress.c b/compress.c\nindex f6986c3..0d0b9d9 100644\n--- a/compress.c\n+++ b/compress.c\n@@ -12,7 +12,7 @@ int compress_start(z_stream *stream,\n                    unsigned char *in, unsigned long in_size,\n                    unsigned char *out, unsigned long out_size)\n {\n-\tstream->next_out = (out ? out : xmalloc(out_size));\n+\tstream->next_out = out;\n \tstream->avail_out = out_size;\n \tstream->next_in = in;\n \tstream->avail_in = in_size;\n@@ -36,19 +36,18 @@ unsigned long compress_free(z_stream *stream)\n \treturn stream->total_out;\n }\n \n-unsigned long compress_all(int level, unsigned char *data,\n-                           unsigned long size, unsigned char **out)\n+unsigned long compress_all(int level, unsigned char *in,\n+                           unsigned long in_size, unsigned char **out)\n {\n-\tint bound, result;\n+\tunsigned long out_size;\n \tz_stream stream;\n \n-\tbound = compress_alloc(&stream, level, size);\n-\tcompress_start(&stream, data, size, NULL, bound);\n+\tout_size = compress_alloc(&stream, level, in_size);\n+\t*out = xmalloc(out_size);\n \n-\t*out = stream.next_out;\n-\tresult = compress_next(&stream, Z_FINISH);\n-\n-\tif (result != Z_STREAM_END) {\n+\tif (   compress_start(&stream, in, in_size, *out, out_size) != Z_OK\n+\t    || compress_next(&stream, Z_FINISH) != Z_STREAM_END)\n+\t{\n \t\tcompress_free(&stream);\n \t\tfree(*out);\n \t\t*out = NULL;\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67129","messageId":"1201952156-6764-7-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-6-git-send-email-mcostalba@gmail.com","subject":"[PATCH 07/11] Introduce stream decompress helpers","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:52Z","receivedAt":"2008-02-02T11:35:52Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Decompressing turns out to be more difficult then\ncomrpessing.\n\nHelpers are more because more are the way\nzlib deflate() is used in git.\n\nThis patch just introduces the helpers,\nstill no code change.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n compress.c |   81 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++\n compress.h |   17 ++++++++++++-\n 2 files changed, 97 insertions(+), 1 deletions(-)\n\ndiff --git a/compress.c b/compress.c\nindex 0d0b9d9..cf9d5ca 100644\n--- a/compress.c\n+++ b/compress.c\n@@ -1,6 +1,10 @@\n #include \"cache.h\"\n #include \"compress.h\"\n \n+/*\n+ *     Compression helpers\n+ */\n+\n unsigned long compress_alloc(z_stream *stream, int level, unsigned long size)\n {\n \tmemset(stream, 0, sizeof(*stream));\n@@ -55,3 +59,80 @@ unsigned long compress_all(int level, unsigned char *in,\n \t}\n \treturn compress_free(&stream);\n }\n+\n+\n+/*\n+ *     Decompression helpers\n+ */\n+\n+int decompress_alloc(z_stream *stream)\n+{\n+\tmemset(stream, 0, sizeof(*stream));\n+\treturn inflateInit(stream);\n+}\n+\n+int decompress_from(z_stream *stream, unsigned char *in, unsigned long in_size)\n+{\n+\tstream->next_in = in;\n+\tstream->avail_in = in_size;\n+\treturn Z_OK;\n+}\n+\n+int decompress_into(z_stream *stream, unsigned char *out, unsigned long out_size)\n+{\n+\tstream->next_out = out;\n+\tstream->avail_out = out_size;\n+\treturn Z_OK;\n+}\n+\n+int decompress_next(z_stream *stream, int flush)\n+{\n+\treturn inflate(stream, flush);\n+}\n+\n+int decompress_next_from(z_stream *stream, unsigned char *in, unsigned long in_size, int flush)\n+{\n+\tdecompress_from(stream, in, in_size);\n+\treturn inflate(stream, flush);\n+}\n+\n+int decompress_next_into(z_stream *stream, unsigned char *out, unsigned long out_size, int flush)\n+{\n+\tdecompress_into(stream, out, out_size);\n+\treturn inflate(stream, flush);\n+}\n+\n+unsigned long decompress_free(z_stream *stream)\n+{\n+\tinflateEnd(stream);\n+\treturn stream->total_out;\n+}\n+\n+unsigned long decompress_all(unsigned char *in, unsigned long in_size,\n+                             unsigned char *out, unsigned long out_size)\n+{\n+/* caller should check for return value != 0 */\n+\n+\tz_stream stream;\n+\tint st;\n+\n+\tif (decompress_alloc(&stream) != Z_OK)\n+\t\treturn 0;\n+\n+\tif (   decompress_from(&stream, in, in_size) != Z_OK\n+\t    || decompress_into(&stream, out, out_size) != Z_OK)\n+\t\tgoto fail;\n+\n+\tdo {\n+\t\tst = decompress_next(&stream, Z_FINISH);\n+\t} while (st == Z_OK);\n+\n+\tif (st != Z_STREAM_END)\n+\t\tgoto fail;\n+\n+\treturn decompress_free(&stream);\n+\n+fail:\n+\tdecompress_free(&stream);\n+\treturn 0;\n+}\ndiff --git a/compress.h b/compress.h\nindex d73c365..30cc80f 100644\n--- a/compress.h\n+++ b/compress.h\n@@ -6,7 +6,22 @@ extern int compress_start(z_stream *stream, unsigned char *in, unsigned long in_\n                            unsigned char *out, unsigned long out_size);\n extern int compress_next(z_stream *stream, int flush);\n extern unsigned long compress_free(z_stream *stream);\n-extern unsigned long compress_all(int level, unsigned char *data, unsigned long size,\n+extern unsigned long compress_all(int level, unsigned char *in, unsigned long in_size,\n                                   unsigned char **out);\n \n+\n+extern int decompress_alloc(z_stream *stream);\n+\n+extern int decompress_from(z_stream *stream, unsigned char *in, unsigned long in_size);\n+extern int decompress_into(z_stream *stream, unsigned char *out, unsigned long out_size);\n+\n+extern int decompress_next(z_stream *stream, int flush);\n+extern int decompress_next_from(z_stream *stream, unsigned char *in, unsigned long in_size, int flush);\n+extern int decompress_next_into(z_stream *stream, unsigned char *out, unsigned long out_size, int flush);\n+\n+extern unsigned long decompress_free(z_stream *stream);\n+\n+extern unsigned long decompress_all(unsigned char *in, unsigned long in_size,\n+                                    unsigned char *out, unsigned long out_size);\n+\n #endif\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67128","messageId":"1201952156-6764-8-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-7-git-send-email-mcostalba@gmail.com","subject":"[PATCH 08/11] Use new decompress_all() helper in git","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:53Z","receivedAt":"2008-02-02T11:35:53Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Only in two places is possible to really simplify\ndeflate code with the all_in_one decompress_all()\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n builtin-apply.c |   23 +++++++++--------------\n index-pack.c    |   30 +++++++-----------------------\n 2 files changed, 16 insertions(+), 37 deletions(-)\n\ndiff --git a/builtin-apply.c b/builtin-apply.c\nindex 30d86f2..fa589e6 100644\n--- a/builtin-apply.c\n+++ b/builtin-apply.c\n@@ -8,6 +8,7 @@\n  */\n #include \"cache.h\"\n #include \"cache-tree.h\"\n+#include \"compress.h\"\n #include \"quote.h\"\n #include \"blob.h\"\n #include \"delta.h\"\n@@ -1105,23 +1106,17 @@ static inline int metadata_changes(struct patch *patch)\n static char *inflate_it(const void *data, unsigned long size,\n \t\t\tunsigned long inflated_size)\n {\n-\tz_stream stream;\n-\tvoid *out;\n-\tint st;\n+\tunsigned char *out;\n+\tunsigned long out_size;\n \n-\tmemset(&stream, 0, sizeof(stream));\n+\tout = xmalloc(inflated_size);\n+\tout_size = decompress_all((unsigned char *)data, size, out, inflated_size);\n \n-\tstream.next_in = (unsigned char *)data;\n-\tstream.avail_in = size;\n-\tstream.next_out = out = xmalloc(inflated_size);\n-\tstream.avail_out = inflated_size;\n-\tinflateInit(&stream);\n-\tst = inflate(&stream, Z_FINISH);\n-\tif ((st != Z_STREAM_END) || stream.total_out != inflated_size) {\n+\tif (out_size != inflated_size) {\n \t\tfree(out);\n \t\treturn NULL;\n \t}\n-\treturn out;\n+\treturn (char *)out;\n }\n \n static struct fragment *parse_binary_hunk(char **buf_p,\ndiff --git a/index-pack.c b/index-pack.c\nindex 880088e..30d7837 100644\n--- a/index-pack.c\n+++ b/index-pack.c\n@@ -169,24 +169,18 @@ static void *unpack_entry_data(unsigned long offset, unsigned long size)\n \tz_stream stream;\n \tvoid *buf = xmalloc(size);\n \n-\tmemset(&stream, 0, sizeof(stream));\n-\tstream.next_out = buf;\n-\tstream.avail_out = size;\n-\tstream.next_in = fill(1);\n-\tstream.avail_in = input_len;\n-\tinflateInit(&stream);\n+\tdecompress_alloc(&stream);\n+\tdecompress_into(&stream, buf, size);\n \n \tfor (;;) {\n-\t\tint ret = inflate(&stream, 0);\n+\t\tint ret = decompress_next_from(&stream, fill(1), input_len, Z_NO_FLUSH);\n \t\tuse(input_len - stream.avail_in);\n \t\tif (stream.total_out == size && ret == Z_STREAM_END)\n \t\t\tbreak;\n \t\tif (ret != Z_OK)\n-\t\t\tbad_object(offset, \"inflate returned %d\", ret);\n-\t\tstream.next_in = fill(1);\n-\t\tstream.avail_in = input_len;\n+\t\t\tbad_object(offset, \"decompress returned %d\", ret);\n \t}\n-\tinflateEnd(&stream);\n+\tdecompress_free(&stream);\n \treturn buf;\n }\n \n@@ -261,8 +255,6 @@ static void *get_data_from_pack(struct object_entry *obj)\n \tunsigned long len = obj[1].idx.offset - from;\n \tunsigned long rdy = 0;\n \tunsigned char *src, *data;\n-\tz_stream stream;\n-\tint st;\n \n \tsrc = xmalloc(len);\n \tdata = src;\n@@ -273,16 +265,8 @@ static void *get_data_from_pack(struct object_entry *obj)\n \t\trdy += n;\n \t} while (rdy < len);\n \tdata = xmalloc(obj->size);\n-\tmemset(&stream, 0, sizeof(stream));\n-\tstream.next_out = data;\n-\tstream.avail_out = obj->size;\n-\tstream.next_in = src;\n-\tstream.avail_in = len;\n-\tinflateInit(&stream);\n-\twhile ((st = inflate(&stream, Z_FINISH)) == Z_OK);\n-\tinflateEnd(&stream);\n-\tif (st != Z_STREAM_END || stream.total_out != obj->size)\n-\t\tdie(\"serious inflate inconsistency\");\n+\tif (decompress_all(src, len, data, obj->size) != obj->size)\n+\t\tdie(\"serious decompress inconsistency\");\n \tfree(src);\n \treturn data;\n }\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67126","messageId":"1201952156-6764-9-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-8-git-send-email-mcostalba@gmail.com","subject":"[PATCH 09/11] Convert http-push.c and http-walker.c","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:54Z","receivedAt":"2008-02-02T11:35:54Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Conversion for both files is very similar\nand in both cases I have added a FIXME where\nI would have added an additional decompress_free()\n\nThe corresponding deflateEnd() call is not present in\nthe original code, so I left the line commented out.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n http-push.c   |   21 +++++++++------------\n http-walker.c |   22 +++++++++-------------\n 2 files changed, 18 insertions(+), 25 deletions(-)\n\ndiff --git a/http-push.c b/http-push.c\nindex a7997ec..0920640 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -203,12 +203,11 @@ static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb,\n \t\tposn += retval;\n \t} while (posn < size);\n \n-\trequest->stream.avail_in = size;\n-\trequest->stream.next_in = ptr;\n+\tdecompress_from(&request->stream, ptr, size);\n+\n \tdo {\n-\t\trequest->stream.next_out = expn;\n-\t\trequest->stream.avail_out = sizeof(expn);\n-\t\trequest->zret = inflate(&request->stream, Z_SYNC_FLUSH);\n+\t\trequest->zret = decompress_next_into(&request->stream, expn,\n+\t\t                                     sizeof(expn), Z_SYNC_FLUSH);\n \t\tSHA1_Update(&request->c, expn,\n \t\t\t    sizeof(expn) - request->stream.avail_out);\n \t} while (request->stream.avail_in && request->zret == Z_OK);\n@@ -266,9 +265,7 @@ static void start_fetch_loose(struct transfer_request *request)\n \t\treturn;\n \t}\n \n-\tmemset(&request->stream, 0, sizeof(request->stream));\n-\n-\tinflateInit(&request->stream);\n+\tdecompress_alloc(&request->stream);\n \n \tSHA1_Init(&request->c);\n \n@@ -305,11 +302,11 @@ static void start_fetch_loose(struct transfer_request *request)\n \t}\n \tunlink(prevfile);\n \n-\t/* Reset inflate/SHA1 if there was an error reading the previous temp\n+\t/* Reset decompress/SHA1 if there was an error reading the previous temp\n \t   file; also rewind to the beginning of the local file. */\n \tif (prev_read == -1) {\n-\t\tmemset(&request->stream, 0, sizeof(request->stream));\n-\t\tinflateInit(&request->stream);\n+\t\t// FIXME should we need decompress_free() here?\n+\t\tdecompress_alloc(&request->stream);\n \t\tSHA1_Init(&request->c);\n \t\tif (prev_posn>0) {\n \t\t\tprev_posn = 0;\n@@ -735,7 +732,7 @@ static void finish_request(struct transfer_request *request)\n \t\t\tif (request->http_code == 416)\n \t\t\t\tfprintf(stderr, \"Warning: requested range invalid; we may already have all the data.\\n\");\n \n-\t\t\tinflateEnd(&request->stream);\n+\t\t\tdecompress_free(&request->stream);\n \t\t\tSHA1_Final(request->real_sha1, &request->c);\n \t\t\tif (request->zret != Z_STREAM_END) {\n \t\t\t\tunlink(request->tmpfile);\ndiff --git a/http-walker.c b/http-walker.c\nindex 2c37868..b1d2a28 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -1,5 +1,6 @@\n #include \"cache.h\"\n #include \"commit.h\"\n+#include \"compress.h\"\n #include \"pack.h\"\n #include \"walker.h\"\n #include \"http.h\"\n@@ -77,12 +78,10 @@ static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb,\n \t\tposn += retval;\n \t} while (posn < size);\n \n-\tobj_req->stream.avail_in = size;\n-\tobj_req->stream.next_in = ptr;\n+\tdecompress_from(&obj_req->stream, ptr, size);\n \tdo {\n-\t\tobj_req->stream.next_out = expn;\n-\t\tobj_req->stream.avail_out = sizeof(expn);\n-\t\tobj_req->zret = inflate(&obj_req->stream, Z_SYNC_FLUSH);\n+\t\tobj_req->zret = decompress_next_into(&obj_req->stream, expn,\n+                                             sizeof(expn), Z_SYNC_FLUSH);\n \t\tSHA1_Update(&obj_req->c, expn,\n \t\t\t    sizeof(expn) - obj_req->stream.avail_out);\n \t} while (obj_req->stream.avail_in && obj_req->zret == Z_OK);\n@@ -140,10 +139,7 @@ static void start_object_request(struct walker *walker,\n \t\treturn;\n \t}\n \n-\tmemset(&obj_req->stream, 0, sizeof(obj_req->stream));\n-\n-\tinflateInit(&obj_req->stream);\n-\n+\tdecompress_alloc(&obj_req->stream);\n \tSHA1_Init(&obj_req->c);\n \n \turl = xmalloc(strlen(obj_req->repo->base) + 51);\n@@ -179,11 +175,11 @@ static void start_object_request(struct walker *walker,\n \t}\n \tunlink(prevfile);\n \n-\t/* Reset inflate/SHA1 if there was an error reading the previous temp\n+\t/* Reset decompress/SHA1 if there was an error reading the previous temp\n \t   file; also rewind to the beginning of the local file. */\n \tif (prev_read == -1) {\n-\t\tmemset(&obj_req->stream, 0, sizeof(obj_req->stream));\n-\t\tinflateInit(&obj_req->stream);\n+\t\t// FIXME should we need decompress_free() here?\n+\t\tdecompress_alloc(&obj_req->stream);\n \t\tSHA1_Init(&obj_req->c);\n \t\tif (prev_posn>0) {\n \t\t\tprev_posn = 0;\n@@ -243,7 +239,7 @@ static void finish_object_request(struct object_request *obj_req)\n \t\treturn;\n \t}\n \n-\tinflateEnd(&obj_req->stream);\n+\tdecompress_free(&obj_req->stream);\n \tSHA1_Final(obj_req->real_sha1, &obj_req->c);\n \tif (obj_req->zret != Z_STREAM_END) {\n \t\tunlink(obj_req->tmpfile);\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67124","messageId":"1201952156-6764-10-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-9-git-send-email-mcostalba@gmail.com","subject":"[PATCH 10/11] Convert builtin-pack/unpack","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:55Z","receivedAt":"2008-02-02T11:35:55Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"In this case decompression helper conversion is\nquite similar and not too complex, so they go\ntogheter.\n\nAlso in index-pack.c pass correct arguments\nto decompress_next_from().\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n builtin-pack-objects.c   |   14 ++++++--------\n builtin-unpack-objects.c |   22 +++++++++-------------\n index-pack.c             |    4 +++-\n 3 files changed, 18 insertions(+), 22 deletions(-)\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex 991a30f..43614ce 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -302,19 +302,17 @@ static int check_pack_inflate(struct packed_git *p,\n {\n \tz_stream stream;\n \tunsigned char fakebuf[4096], *in;\n+\tunsigned int in_size = 0;\n \tint st;\n \n-\tmemset(&stream, 0, sizeof(stream));\n-\tinflateInit(&stream);\n+\tdecompress_alloc(&stream);\n \tdo {\n-\t\tin = use_pack(p, w_curs, offset, &stream.avail_in);\n-\t\tstream.next_in = in;\n-\t\tstream.next_out = fakebuf;\n-\t\tstream.avail_out = sizeof(fakebuf);\n-\t\tst = inflate(&stream, Z_FINISH);\n+\t\tdecompress_into(&stream, fakebuf, sizeof(fakebuf));\n+\t\tin = use_pack(p, w_curs, offset, &in_size);\n+\t\tst = decompress_next_from(&stream, in, in_size, Z_FINISH);\n \t\toffset += stream.next_in - in;\n \t} while (st == Z_OK || st == Z_BUF_ERROR);\n-\tinflateEnd(&stream);\n+\tdecompress_free(&stream);\n \treturn (st == Z_STREAM_END &&\n \t\tstream.total_out == expect &&\n \t\tstream.total_in == len) ? 0 : -1;\ndiff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c\nindex 1e51865..c996560 100644\n--- a/builtin-unpack-objects.c\n+++ b/builtin-unpack-objects.c\n@@ -1,5 +1,6 @@\n #include \"builtin.h\"\n #include \"cache.h\"\n+#include \"compress.h\"\n #include \"object.h\"\n #include \"delta.h\"\n #include \"pack.h\"\n@@ -61,23 +62,20 @@ static void use(int bytes)\n static void *get_data(unsigned long size)\n {\n \tz_stream stream;\n-\tvoid *buf = xmalloc(size);\n+\tunsigned char *buf = xmalloc(size);;\n \n-\tmemset(&stream, 0, sizeof(stream));\n-\n-\tstream.next_out = buf;\n-\tstream.avail_out = size;\n-\tstream.next_in = fill(1);\n-\tstream.avail_in = len;\n-\tinflateInit(&stream);\n+\tdecompress_alloc(&stream);\n+\tdecompress_into(&stream, buf, size);\n \n \tfor (;;) {\n-\t\tint ret = inflate(&stream, 0);\n+\t\t/* fill() modifies len, so be sure is evaluated as first */\n+\t\tvoid* tmp = fill(1);\n+\t\tint ret = decompress_next_from(&stream, tmp, len, Z_NO_FLUSH);\n \t\tuse(len - stream.avail_in);\n \t\tif (stream.total_out == size && ret == Z_STREAM_END)\n \t\t\tbreak;\n \t\tif (ret != Z_OK) {\n-\t\t\terror(\"inflate returned %d\\n\", ret);\n+\t\t\terror(\"decompress returned %d\\n\", ret);\n \t\t\tfree(buf);\n \t\t\tbuf = NULL;\n \t\t\tif (!recover)\n@@ -85,10 +83,8 @@ static void *get_data(unsigned long size)\n \t\t\thas_errors = 1;\n \t\t\tbreak;\n \t\t}\n-\t\tstream.next_in = fill(1);\n-\t\tstream.avail_in = len;\n \t}\n-\tinflateEnd(&stream);\n+\tdecompress_free(&stream);\n \treturn buf;\n }\n \ndiff --git a/index-pack.c b/index-pack.c\nindex 30d7837..929de39 100644\n--- a/index-pack.c\n+++ b/index-pack.c\n@@ -173,7 +173,9 @@ static void *unpack_entry_data(unsigned long offset, unsigned long size)\n \tdecompress_into(&stream, buf, size);\n \n \tfor (;;) {\n-\t\tint ret = decompress_next_from(&stream, fill(1), input_len, Z_NO_FLUSH);\n+\t\t/* fill() modifies len, so be sure is evaluated as first */\n+\t\tvoid* tmp = fill(1);\n+\t\tint ret = decompress_next_from(&stream, tmp, input_len, Z_NO_FLUSH);\n \t\tuse(input_len - stream.avail_in);\n \t\tif (stream.total_out == size && ret == Z_STREAM_END)\n \t\t\tbreak;\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67125","messageId":"1201952156-6764-11-git-send-email-mcostalba@gmail.com","threadId":"11839","inReplyTo":"1201952156-6764-10-git-send-email-mcostalba@gmail.com","subject":"[PATCH 11/11] Convert sha1_file.c to use decompress helpers","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-02-02T11:35:56Z","receivedAt":"2008-02-02T11:35:56Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"This is \"The King\".\n\nIt is the most difficult file to convert and some\ndecompression functions have been created just for it.\n\nAnyhow the lines of code removed (45) far surpass the\nones added (26).\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n sha1_file.c |   71 +++++++++++++++++++++-------------------------------------\n 1 files changed, 26 insertions(+), 45 deletions(-)\n\ndiff --git a/sha1_file.c b/sha1_file.c\nindex f48ad04..6500871 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1079,16 +1079,11 @@ static int unpack_sha1_header(z_stream *stream, unsigned char *map, unsigned lon\n \tenum object_type type;\n \n \t/* Get the data stream */\n-\tmemset(stream, 0, sizeof(*stream));\n-\tstream->next_in = map;\n-\tstream->avail_in = mapsize;\n-\tstream->next_out = buffer;\n-\tstream->avail_out = bufsiz;\n+\tdecompress_alloc(stream);\n+\tdecompress_into(stream, buffer, bufsiz);\n \n-\tif (legacy_loose_object(map)) {\n-\t\tinflateInit(stream);\n-\t\treturn inflate(stream, 0);\n-\t}\n+\tif (legacy_loose_object(map))\n+\t\treturn decompress_next_from(stream, map, mapsize, Z_NO_FLUSH);\n \n \n \t/*\n@@ -1105,9 +1100,7 @@ static int unpack_sha1_header(z_stream *stream, unsigned char *map, unsigned lon\n \tmapsize -= used;\n \n \t/* Set up the stream for the rest.. */\n-\tstream->next_in = map;\n-\tstream->avail_in = mapsize;\n-\tinflateInit(stream);\n+\tdecompress_from(stream, map, mapsize);\n \n \t/* And generate the fake traditional header */\n \tstream->total_out = 1 + snprintf(buffer, bufsiz, \"%s %lu\",\n@@ -1141,14 +1134,13 @@ static void *unpack_sha1_rest(z_stream *stream, void *buffer, unsigned long size\n \t\t * we also want to check that zlib tells us that all\n \t\t * went well with status == Z_STREAM_END at the end.\n \t\t */\n-\t\tstream->next_out = buf + bytes;\n-\t\tstream->avail_out = size - bytes;\n+\t\tdecompress_into(stream, buf + bytes, size - bytes);\n \t\twhile (status == Z_OK)\n-\t\t\tstatus = inflate(stream, Z_FINISH);\n+\t\t\tstatus = decompress_next(stream, Z_FINISH);\n \t}\n \tbuf[size] = 0;\n \tif (status == Z_STREAM_END && !stream->avail_in) {\n-\t\tinflateEnd(stream);\n+\t\tdecompress_free(stream);\n \t\treturn buf;\n \t}\n \n@@ -1233,20 +1225,18 @@ unsigned long get_size_from_delta(struct packed_git *p,\n \tunsigned char delta_head[20], *in;\n \tz_stream stream;\n \tint st;\n+\tunsigned int in_size = 0;\n \n-\tmemset(&stream, 0, sizeof(stream));\n-\tstream.next_out = delta_head;\n-\tstream.avail_out = sizeof(delta_head);\n+\tdecompress_alloc(&stream);\n+\tdecompress_into(&stream, delta_head, sizeof(delta_head));\n \n-\tinflateInit(&stream);\n \tdo {\n-\t\tin = use_pack(p, w_curs, curpos, &stream.avail_in);\n-\t\tstream.next_in = in;\n-\t\tst = inflate(&stream, Z_FINISH);\n+\t\tin = use_pack(p, w_curs, curpos, &in_size);\n+\t\tst = decompress_next_from(&stream, in, in_size, Z_FINISH);\n \t\tcurpos += stream.next_in - in;\n \t} while ((st == Z_OK || st == Z_BUF_ERROR) &&\n \t\t stream.total_out < sizeof(delta_head));\n-\tinflateEnd(&stream);\n+\tdecompress_free(&stream);\n \tif ((st != Z_STREAM_END) && stream.total_out != sizeof(delta_head))\n \t\tdie(\"delta data unpack-initial failed\");\n \n@@ -1323,7 +1313,7 @@ static int packed_delta_info(struct packed_git *p,\n \t/* We choose to only get the type of the base object and\n \t * ignore potentially corrupt pack file that expects the delta\n \t * based on a base with a wrong size.  This saves tons of\n-\t * inflate() calls.\n+\t * decompress() calls.\n \t */\n \tif (sizep)\n \t\t*sizep = get_size_from_delta(p, w_curs, curpos);\n@@ -1444,21 +1434,18 @@ static void *unpack_compressed_entry(struct packed_git *p,\n \tint st;\n \tz_stream stream;\n \tunsigned char *buffer, *in;\n+\tunsigned int in_size = 0;\n \n \tbuffer = xmalloc(size + 1);\n \tbuffer[size] = 0;\n-\tmemset(&stream, 0, sizeof(stream));\n-\tstream.next_out = buffer;\n-\tstream.avail_out = size;\n-\n-\tinflateInit(&stream);\n+\tdecompress_alloc(&stream);\n+\tdecompress_into(&stream, buffer, size);\n \tdo {\n-\t\tin = use_pack(p, w_curs, curpos, &stream.avail_in);\n-\t\tstream.next_in = in;\n-\t\tst = inflate(&stream, Z_FINISH);\n+\t\tin = use_pack(p, w_curs, curpos, &in_size);\n+\t\tst = decompress_next_from(&stream, in, in_size, Z_FINISH);\n \t\tcurpos += stream.next_in - in;\n \t} while (st == Z_OK || st == Z_BUF_ERROR);\n-\tinflateEnd(&stream);\n+\tdecompress_free(&stream);\n \tif ((st != Z_STREAM_END) || stream.total_out != size) {\n \t\tfree(buffer);\n \t\treturn NULL;\n@@ -1804,7 +1791,7 @@ static int sha1_loose_object_info(const unsigned char *sha1, unsigned long *size\n \t\tstatus = error(\"unable to parse %s header\", sha1_to_hex(sha1));\n \telse if (sizep)\n \t\t*sizep = size;\n-\tinflateEnd(&stream);\n+\tdecompress_free(&stream);\n \tmunmap(map, mapsize);\n \treturn status;\n }\n@@ -2212,21 +2199,15 @@ int write_sha1_from_fd(const unsigned char *sha1, int fd, char *buffer,\n \t\t\treturn error(\"unable to create temporary sha1 filename %s: %s\\n\", tmpfile, strerror(errno));\n \t}\n \n-\tmemset(&stream, 0, sizeof(stream));\n-\n-\tinflateInit(&stream);\n-\n+\tdecompress_alloc(&stream);\n \tSHA1_Init(&c);\n \n \tdo {\n \t\tssize_t size;\n \t\tif (*bufposn) {\n-\t\t\tstream.avail_in = *bufposn;\n-\t\t\tstream.next_in = (unsigned char *) buffer;\n+\t\t\tdecompress_from(&stream, (unsigned char *) buffer, *bufposn);\n \t\t\tdo {\n-\t\t\t\tstream.next_out = discard;\n-\t\t\t\tstream.avail_out = sizeof(discard);\n-\t\t\t\tret = inflate(&stream, Z_SYNC_FLUSH);\n+\t\t\t\tret = decompress_next_into(&stream, discard, sizeof(discard), Z_SYNC_FLUSH);\n \t\t\t\tSHA1_Update(&c, discard, sizeof(discard) -\n \t\t\t\t\t    stream.avail_out);\n \t\t\t} while (stream.avail_in && ret == Z_OK);\n@@ -2249,7 +2230,7 @@ int write_sha1_from_fd(const unsigned char *sha1, int fd, char *buffer,\n \t\t}\n \t\t*bufposn += size;\n \t} while (1);\n-\tinflateEnd(&stream);\n+\tdecompress_free(&stream);\n \n \tfchmod(local, 0444);\n \tif (close(local) != 0)\n-- \n1.5.4.rc4.39.g524a\n"},{"id":"67273","messageId":"7v8x21slck.fsf@gitster.siamese.dyndns.org","threadId":"11839","inReplyTo":"1201952156-6764-1-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 01/11] Introduce stream compress helpers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-03T22:53:31Z","receivedAt":"2008-02-03T22:53:31Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marco Costalba <mcostalba@gmail.com> writes:\n\n> When decompressing a zlib stream use this\n> helpers instead of calling low level zlib\n> function.\n>\n> This patch introduces the necessary framework,\n> still no code change.\n>\n> This is the first step in generalizing compress and\n> decompress functions avoiding zlib directly calls.\n\n(Cosmetic)\n\nYour log message lines are wrapped a bit too short, while some\nof the patch lines are too long.\n\n> diff --git a/compress.c b/compress.c\n> new file mode 100644\n> index 0000000..f6986c3\n> --- /dev/null\n> +++ b/compress.c\n> @@ -0,0 +1,58 @@\n> +#include \"cache.h\"\n> +#include \"compress.h\"\n> +\n> +unsigned long compress_alloc(z_stream *stream, int level, unsigned long size)\n> +{\n> +\tmemset(stream, 0, sizeof(*stream));\n> +\tdeflateInit(stream, level);\n> +\treturn deflateBound(stream, size);\n> +}\n\n(Naming)\n\nThis is not about \"allocation\", but about \"setup\".\n\n> +int compress_start(z_stream *stream,\n> +                   unsigned char *in, unsigned long in_size,\n> +                   unsigned char *out, unsigned long out_size)\n> +{\n> +\tstream->next_out = (out ? out : xmalloc(out_size));\n> +\tstream->avail_out = out_size;\n> +\tstream->next_in = in;\n> +\tstream->avail_in = in_size;\n> +\treturn Z_OK;\n> +}\n\nThis returns Z_OK unconditionally and most callers do not even\nbother checking the return value.  Shouldn't this be of type\nvoid?\n\nEspecially the use of this in if() conditional, after [06/11]\nchanges its use in compress_all(), looks quite ugly.\n\n> +unsigned long compress_free(z_stream *stream)\n> +{\n> +\tdeflateEnd(stream);\n> +\treturn stream->total_out;\n> +}\n\nEventually, this should check errors from deflateEnd() and\npropagate that to the caller.\n"},{"id":"67274","messageId":"7v3as9slc6.fsf@gitster.siamese.dyndns.org","threadId":"11839","inReplyTo":"1201952156-6764-3-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 03/11] Use new compress helpers in fast-import","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-03T22:53:45Z","receivedAt":"2008-02-03T22:53:45Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marco Costalba <mcostalba@gmail.com> writes:\n\n> Here is slightly more difficult, in particular\n> a xrealloc() has been substituted with a\n> free() + xmalloc() to keep the code simple.\n>\n> Signed-off-by: Marco Costalba <mcostalba@gmail.com>\n> ---\n>  fast-import.c |   45 +++++++++++++++------------------------------\n>  1 files changed, 15 insertions(+), 30 deletions(-)\n\nI'll let Shawn comment on this.  The realloc() does not seem to\nbe using the contents in the buffer from the previous round, so\nI suspect that a free() followed by an independent alloc() would\nbe an improvement when the later call uses much larger buffer\nthan the previous one, but would be a waste if the later one\nneeds smaller buffer.\n"},{"id":"67275","messageId":"7vwsplr6re.fsf@gitster.siamese.dyndns.org","threadId":"11839","inReplyTo":"1201952156-6764-4-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 04/11] Use new compress helpers in http-push.c","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-03T22:53:57Z","receivedAt":"2008-02-03T22:53:57Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marco Costalba <mcostalba@gmail.com> writes:\n\n> A multistep compress is required here, so\n> we need the full arsenal of compress helpers.\n>\n> Signed-off-by: Marco Costalba <mcostalba@gmail.com>\n> ---\n>  http-push.c |   22 ++++++++--------------\n>  1 files changed, 8 insertions(+), 14 deletions(-)\n>\n> diff --git a/http-push.c b/http-push.c\n> index b2b410d..a7997ec 100644\n> --- a/http-push.c\n> +++ b/http-push.c\n> @@ -491,31 +492,24 @@ static void start_put(struct transfer_request *request)\n> ...\n> -\tstream.next_in = (void *)hdr;\n> -\tstream.avail_in = hdrlen;\n> -\twhile (deflate(&stream, 0) == Z_OK)\n> -\t\t/* nothing */;\n> +\tcompress_next(&stream, Z_NO_FLUSH);\n\nAlthough the original does not bother to, the return value from\ncompress_next() should be checked in later rounds.\n"},{"id":"67276","messageId":"7vr6ftr6qu.fsf@gitster.siamese.dyndns.org","threadId":"11839","inReplyTo":"1201952156-6764-2-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 02/11] Use new compress helpers in git files","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-03T22:54:17Z","receivedAt":"2008-02-03T22:54:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marco Costalba <mcostalba@gmail.com> writes:\n\n> These are the 'easy' ones, where a signgle step\n> compression is requested so that we can use only\n> one call to compress_all()\n>\n> Signed-off-by: Marco Costalba <mcostalba@gmail.com>\n> ---\n>  archive-zip.c          |   28 +++-------------------------\n>  builtin-pack-objects.c |   19 +++----------------\n>  diff.c                 |   22 +++++-----------------\n>  index-pack.c           |   20 +++-----------------\n>  4 files changed, 14 insertions(+), 75 deletions(-)\n>\n> diff --git a/archive-zip.c b/archive-zip.c\n> index 74e30f6..9071b86 100644\n> --- a/archive-zip.c\n> +++ b/archive-zip.c\n> @@ -3,6 +3,7 @@\n>   */\n>  #include \"cache.h\"\n>  #include \"commit.h\"\n> +#include \"compress.h\"\n>  #include \"blob.h\"\n>  #include \"tree.h\"\n>  #include \"quote.h\"\n> @@ -97,33 +98,10 @@ static void copy_le32(unsigned char *dest, unsigned int n)\n>  static void *zlib_deflate(void *data, unsigned long size,\n>                            unsigned long *compressed_size)\n>  {\n> ...\n> -\tif (result != Z_STREAM_END) {\n> -\t\tfree(buffer);\n> -\t\treturn NULL;\n> -\t}\n>  \n> -\tdeflateEnd(&stream);\n> -\t*compressed_size = stream.total_out;\n> +\tunsigned char *buffer = NULL;\n>  \n> +\t*compressed_size = compress_all(zlib_compression_level, data, size, &buffer);\n>  \treturn buffer;\n>  }\n\nThis used to leave *compressed_size untouched upon deflate\nfailure but now it sets it to zero.  Is this change in behaviour\nsafe?\n"},{"id":"67277","messageId":"7vlk61r6qk.fsf@gitster.siamese.dyndns.org","threadId":"11839","inReplyTo":"1201952156-6764-6-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 06/11] Better error handling in compress_all()","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-03T22:54:27Z","receivedAt":"2008-02-03T22:54:27Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marco Costalba <mcostalba@gmail.com> writes:\n\n> Also let the caller to xmalloc() the buffer\n> int compress_start()\n\nThis is meant to be an improvement for [01/11] and I think\nshould be done from the beginning by squashing into it.\n\nHaven't looked at the decompression side yet.  Help in reviewing\nthis series from others are appreciated.\n"},{"id":"67303","messageId":"20080204014120.GF24004@spearce.org","threadId":"11839","inReplyTo":"1201952156-6764-3-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 03/11] Use new compress helpers in fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-02-04T01:41:20Z","receivedAt":"2008-02-04T01:41:20Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Marco Costalba <mcostalba@gmail.com> wrote:\n> diff --git a/fast-import.c b/fast-import.c\n> index a523b17..b6bb84c 100644\n> --- a/fast-import.c\n> +++ b/fast-import.c\n> @@ -997,13 +998,13 @@ static int store_object(\n>  \tunsigned char *sha1out,\n>  \tuintmax_t mark)\n>  {\n> -\tvoid *out, *delta;\n> +\tunsigned char *out, *delta;\n>  \tstruct object_entry *e;\n>  \tunsigned char hdr[96];\n>  \tunsigned char sha1[20];\n>  \tunsigned long hdrlen, deltalen;\n>  \tSHA_CTX c;\n> -\tz_stream s;\n> +\tint out_size;\n\nThis really should be a size_t, shouldn't it?\n  \n-- \nShawn.\n"},{"id":"67305","messageId":"20080204014849.GG24004@spearce.org","threadId":"11839","inReplyTo":"7v3as9slc6.fsf@gitster.siamese.dyndns.org","subject":"Re: [PATCH 03/11] Use new compress helpers in fast-import","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2008-02-04T01:48:50Z","receivedAt":"2008-02-04T01:48:50Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Junio C Hamano <gitster@pobox.com> wrote:\n> Marco Costalba <mcostalba@gmail.com> writes:\n> \n> > Here is slightly more difficult, in particular\n> > a xrealloc() has been substituted with a\n> > free() + xmalloc() to keep the code simple.\n> >\n> > Signed-off-by: Marco Costalba <mcostalba@gmail.com>\n> > ---\n> >  fast-import.c |   45 +++++++++++++++------------------------------\n> >  1 files changed, 15 insertions(+), 30 deletions(-)\n> \n> I'll let Shawn comment on this.  The realloc() does not seem to\n> be using the contents in the buffer from the previous round, so\n> I suspect that a free() followed by an independent alloc() would\n> be an improvement when the later call uses much larger buffer\n> than the previous one, but would be a waste if the later one\n> needs smaller buffer.\n\nJunio is correct, that xrealloc isn't using the contents of the\nbuffer from the last round, which makes any memcpy it might do\ninternally due to movement to a larger buffer an utter waste.\n\nIn this new version we are probably always free'ing a buffer of a\nmuch smaller size than we are then later allocating (or in the old\nversion xrealloc'ing to) because we are switching from a delta to\nfull content.  Its most likely the delta is way smaller, so I'd\nguess the malloc implementation is mostly going to another buffer.\nIn short, Marco's change will most likely do better.\n\nBut this is all academic wanking.  We're talking about this xrealloc\n(or free/xmalloc pair) happening only when we switch packfiles,\nwhich in fast-import is usually every 4 GiB of output.  That's a\n*lot* of data to write.  Who cares how many extra microseconds we\nspend to perform this buffer change; we probably hit it only once\nevery 15-30 minutes, depending on how fast your system is able to\ntransfer 4 GiB of data out of the source and into a packfile.\n\n-- \nShawn.\n"},{"id":"67308","messageId":"7v3as9o4oh.fsf@gitster.siamese.dyndns.org","threadId":"11839","inReplyTo":"1201952156-6764-7-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 07/11] Introduce stream decompress helpers","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-04T02:07:10Z","receivedAt":"2008-02-04T02:07:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Overall, everything looks quite nicely done.\n\nThe same comments as the compression side to \"*_alloc()\" apply.\nPerhaps call them \"setup\" and \"finalize\" or something?\n"},{"id":"67309","messageId":"7vve55mq31.fsf@gitster.siamese.dyndns.org","threadId":"11839","inReplyTo":"1201952156-6764-8-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 08/11] Use new decompress_all() helper in git","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-04T02:07:46Z","receivedAt":"2008-02-04T02:07:46Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marco Costalba <mcostalba@gmail.com> writes:\n\n> Only in two places is possible to really simplify\n> diff --git a/index-pack.c b/index-pack.c\n> index 880088e..30d7837 100644\n> --- a/index-pack.c\n> +++ b/index-pack.c\n> @@ -169,24 +169,18 @@ static void *unpack_entry_data(unsigned long offset, unsigned long size)\n>  \tz_stream stream;\n>  \tvoid *buf = xmalloc(size);\n>  \n> -\tmemset(&stream, 0, sizeof(stream));\n> -\tstream.next_out = buf;\n> -\tstream.avail_out = size;\n> -\tstream.next_in = fill(1);\n> -\tstream.avail_in = input_len;\n> -\tinflateInit(&stream);\n> +\tdecompress_alloc(&stream);\n> +\tdecompress_into(&stream, buf, size);\n>  \n>  \tfor (;;) {\n> -\t\tint ret = inflate(&stream, 0);\n> +\t\tint ret = decompress_next_from(&stream, fill(1), input_len, Z_NO_FLUSH);\n\nThe input_len variable is changed as a side effect of calling\nthe fill() function.  Don't you have the same issue that you\nhandle with [10/11] here?\n"},{"id":"67310","messageId":"7vodaxmq2l.fsf@gitster.siamese.dyndns.org","threadId":"11839","inReplyTo":"1201952156-6764-10-git-send-email-mcostalba@gmail.com","subject":"Re: [PATCH 10/11] Convert builtin-pack/unpack","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2008-02-04T02:08:02Z","receivedAt":"2008-02-04T02:08:02Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Marco Costalba <mcostalba@gmail.com> writes:\n\n> diff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c\n> index 1e51865..c996560 100644\n> --- a/builtin-unpack-objects.c\n> +++ b/builtin-unpack-objects.c\n> @@ -1,5 +1,6 @@\n>  #include \"builtin.h\"\n>  #include \"cache.h\"\n> +#include \"compress.h\"\n>  #include \"object.h\"\n>  #include \"delta.h\"\n>  #include \"pack.h\"\n> @@ -61,23 +62,20 @@ static void use(int bytes)\n>  static void *get_data(unsigned long size)\n>  {\n>  \tz_stream stream;\n> -\tvoid *buf = xmalloc(size);\n> +\tunsigned char *buf = xmalloc(size);;\n\nWhy?  Your other changes (e.g. unpack_entry_data()::index-pack.c\nin [08/11]) left the type of buf as it was, and I think the same\nshould be done here.\n\n> -\tmemset(&stream, 0, sizeof(stream));\n> -\n> -\tstream.next_out = buf;\n> -\tstream.avail_out = size;\n> -\tstream.next_in = fill(1);\n> -\tstream.avail_in = len;\n> -\tinflateInit(&stream);\n> +\tdecompress_alloc(&stream);\n> +\tdecompress_into(&stream, buf, size);\n>  \n>  \tfor (;;) {\n> -\t\tint ret = inflate(&stream, 0);\n> +\t\t/* fill() modifies len, so be sure is evaluated as first */\n> +\t\tvoid* tmp = fill(1);\n> +\t\tint ret = decompress_next_from(&stream, tmp, len, Z_NO_FLUSH);\n\n(Style) that's \"void *tmp\".\n"}]}