{"thread":{"id":"11604","subject":"[PATCH 2/3] Rename z_stream to ext_stream across the sources","startedAt":"2008-01-13T13:24:36Z","lastAt":"2008-01-13T13:24:38Z","messageCount":3,"participants":["Marco Costalba"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"65241","messageId":"1200230678-18188-1-git-send-email-mcostalba@gmail.com","threadId":"11604","inReplyTo":null,"subject":"[PATCH 1/3] Convert compress helpers to use an extended z_stream","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-01-13T13:24:36Z","receivedAt":"2008-01-13T13:24:36Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Instead of passing as argument to compress/decompress\nhelpers the zlib native z_stream create a new extended\none called ext_stream and use this instead.\n\nThis will allow us to associate the specified compress\nalgorithm on a 'per stream' base.\n\nThe patch aims to be as less intrusive as possible\nso that ext_stream is really a (slightly) extended\nz_stream and includes a z_stream member.\n\nChoice to use a z_stream variable instead of a z_stream*\npointer as member is to avoid a xmalloc when a local\next_stream variable is used.\n\nThis patch just introduces ext_stream and modifies the\nhelpers to cope with that. Next patch will rename z_stream\nto ext_stream across the sources. Tough this patch will\nnot compile without the next I splitted the two for easing\nthe review: the interesting part is here, the next is\njust renaming.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n\nThe patch series apply above my last one sent yesterday.\n\nThis patch series is intended more as a RFC then to\nbe applied.\n\nWhile the previous series introduced a cleanup, this one\nextends the compress/decompress helpers framwork to support\ndifferent compress backends.\n\nThe compress backends are associated to each z_stream, it means\nI can have multiple z_stream working with different backends at\nthe same time.\n\nThe rationale for this low level link instead of setting the chosen\ncomrpess library, say, at repository level, is that pulling/pushing\npacks on the net must continue to work with zlib so to not break\nthings and also that, as Junio pointed out, today we don't\nunpackage and repackage the objects that arrive with a git-pull,\nbut store the incoming pack as is.\n\nSo we can have different packs compressed with different backends\nin the same repo. To allow this the compression algorithm info should\nbe linked at the z_stream level. Note that this is _almost_ trasparent\nto the user.\n\n\nFinally this is the first patch series sent with send-mail, so to try\nto overcome the lines wrapping damage of my mailer.\n\nBTW it took more time to setup send-mail and send the patches then to\nwrite them! ;-)\n\n\n compress.c |   64 ++++++++++++++++++++++++++++++------------------------------\n compress.h |   52 ++++++++++++++++++++++++++++++++++++++----------\n 2 files changed, 73 insertions(+), 43 deletions(-)\n\ndiff --git a/compress.c b/compress.c\nindex f73cf2c..b28c389 100644\n--- a/compress.c\n+++ b/compress.c\n@@ -5,46 +5,46 @@\n  *     Compression helpers\n  */\n \n-unsigned long compress_alloc(z_stream *stream, int level, unsigned long size)\n+unsigned long compress_alloc(ext_stream *stream, int level, unsigned long size)\n {\n \tmemset(stream, 0, sizeof(*stream));\n-\tdeflateInit(stream, level);\n-\treturn deflateBound(stream, size);\n+\tdeflateInit(&stream->z, level);\n+\treturn deflateBound(&stream->z, size);\n }\n \n-int compress_start(z_stream *stream,\n+int compress_start(ext_stream *stream,\n                    unsigned char *in, unsigned long in_size,\n                    unsigned char *out, unsigned long out_size)\n {\n-\tstream->next_out = out;\n-\tstream->avail_out = out_size;\n-\tstream->next_in = in;\n-\tstream->avail_in = in_size;\n+\tstream->z.next_out = out;\n+\tstream->z.avail_out = out_size;\n+\tstream->z.next_in = in;\n+\tstream->z.avail_in = in_size;\n \treturn Z_OK;\n }\n \n-int compress_next(z_stream *stream, int flush)\n+int compress_next(ext_stream *stream, int flush)\n {\n \tint result;\n \n \tdo {\n-\t\tresult = deflate(stream, flush);\n+\t\tresult = deflate(&stream->z, flush);\n \t} while (result == Z_OK);\n \n \treturn result;\n }\n \n-unsigned long compress_free(z_stream *stream)\n+unsigned long compress_free(ext_stream *stream)\n {\n-\tdeflateEnd(stream);\n-\treturn stream->total_out;\n+\tdeflateEnd(&stream->z);\n+\treturn stream->z.total_out;\n }\n \n unsigned long compress_all(int level, unsigned char *in,\n                            unsigned long in_size, unsigned char **out)\n {\n \tunsigned long out_size;\n-\tz_stream stream;\n+\text_stream stream;\n \n \tout_size = compress_alloc(&stream, level, in_size);\n \t*out = xmalloc(out_size);\n@@ -65,47 +65,47 @@ unsigned long compress_all(int level, unsigned char *in,\n  *     Decompression helpers\n  */\n \n-int decompress_alloc(z_stream *stream)\n+int decompress_alloc(ext_stream *stream)\n {\n \tmemset(stream, 0, sizeof(*stream));\n-\treturn inflateInit(stream);\n+\treturn inflateInit(&stream->z);\n }\n \n-int decompress_from(z_stream *stream, unsigned char *in, unsigned long in_size)\n+int decompress_from(ext_stream *stream, unsigned char *in, unsigned long in_size)\n {\n-\tstream->next_in = in;\n-\tstream->avail_in = in_size;\n+\tstream->z.next_in = in;\n+\tstream->z.avail_in = in_size;\n \treturn Z_OK;\n }\n \n-int decompress_into(z_stream *stream, unsigned char *out, unsigned long out_size)\n+int decompress_into(ext_stream *stream, unsigned char *out, unsigned long out_size)\n {\n-\tstream->next_out = out;\n-\tstream->avail_out = out_size;\n+\tstream->z.next_out = out;\n+\tstream->z.avail_out = out_size;\n \treturn Z_OK;\n }\n \n-int decompress_next(z_stream *stream, int flush)\n+int decompress_next(ext_stream *stream, int flush)\n {\n-\treturn inflate(stream, flush);\n+\treturn inflate(&stream->z, flush);\n }\n \n-int decompress_next_from(z_stream *stream, unsigned char *in, unsigned long in_size, int flush)\n+int decompress_next_from(ext_stream *stream, unsigned char *in, unsigned long in_size, int flush)\n {\n \tdecompress_from(stream, in, in_size);\n-\treturn inflate(stream, flush);\n+\treturn inflate(&stream->z, flush);\n }\n \n-int decompress_next_into(z_stream *stream, unsigned char *out, unsigned long out_size, int flush)\n+int decompress_next_into(ext_stream *stream, unsigned char *out, unsigned long out_size, int flush)\n {\n \tdecompress_into(stream, out, out_size);\n-\treturn inflate(stream, flush);\n+\treturn inflate(&stream->z, flush);\n }\n \n-unsigned long decompress_free(z_stream *stream)\n+unsigned long decompress_free(ext_stream *stream)\n {\n-\tinflateEnd(stream);\n-\treturn stream->total_out;\n+\tinflateEnd(&stream->z);\n+\treturn stream->z.total_out;\n }\n \n unsigned long decompress_all(unsigned char *in, unsigned long in_size,\n@@ -113,7 +113,7 @@ unsigned long decompress_all(unsigned char *in, unsigned long in_size,\n {\n /* caller should check for return value != 0 */\n \n-\tz_stream stream;\n+\text_stream stream;\n \tint st;\n \n \tif (decompress_alloc(&stream) != Z_OK)\ndiff --git a/compress.h b/compress.h\nindex a81d006..d1de31f 100644\n--- a/compress.h\n+++ b/compress.h\n@@ -1,25 +1,55 @@\n #ifndef COMPRESS_H\n #define COMPRESS_H\n \n-extern unsigned long compress_alloc(z_stream *stream, int level, unsigned long size);\n-extern int compress_start(z_stream *stream, unsigned char *in, unsigned long in_size,\n+/* Any compress/decompress engine must implement all the\n+ * below functions that are modeled after the zlib ones.\n+ */\n+typedef int (*deflateInit_fn_t)(z_stream *stream, int level);\n+typedef int (*deflate_fn_t)(z_stream *stream, int flush);\n+typedef int (*deflateEnd_fn_t)(z_stream *stream);\n+typedef unsigned long (*deflateBound_fn_t)(z_stream *stream, unsigned long size);\n+\n+typedef int (*inflateInit_fn_t)(z_stream *stream);\n+typedef int (*inflate_fn_t)(z_stream *stream, int flush);\n+typedef int (*inflateEnd_fn_t)(z_stream *stream);\n+\n+\n+/* Extended struct used instead of the zlib native to\n+ * call the compress/decompress helpers. It's just a\n+ * thin extension of the zlib native one.\n+ */\n+typedef struct ext_stream_s {\n+\tz_stream z; /* defined in zlib.h to store stream state */\n+\n+\t/* pointers to low level compress library functions */\n+\tdeflateInit_fn_t  deflateInit_fn;\n+\tdeflate_fn_t      deflate_fn;\n+\tdeflateEnd_fn_t   deflateEnd_fn;\n+\tdeflateBound_fn_t deflateBound_fn;\n+\tinflateInit_fn_t  inflateInit_fn;\n+\tinflate_fn_t      inflate_fn;\n+\tinflateEnd_fn_t   inflateEnd_fn;\n+} ext_stream;\n+\n+extern unsigned long compress_alloc(ext_stream *stream, int level, unsigned long size);\n+extern int compress_start(ext_stream *stream, unsigned char *in, unsigned long in_size,\n                            unsigned char *out, unsigned long out_size);\n-extern int compress_next(z_stream *stream, int flush);\n-extern unsigned long compress_free(z_stream *stream);\n+extern int compress_next(ext_stream *stream, int flush);\n+extern unsigned long compress_free(ext_stream *stream);\n extern unsigned long compress_all(int level, unsigned char *in, unsigned long in_size,\n                                   unsigned char **out);\n \n \n-extern int decompress_alloc(z_stream *stream);\n+extern int decompress_alloc(ext_stream *stream);\n \n-extern int decompress_from(z_stream *stream, unsigned char *in, unsigned long in_size);\n-extern int decompress_into(z_stream *stream, unsigned char *out, unsigned long out_size);\n+extern int decompress_from(ext_stream *stream, unsigned char *in, unsigned long in_size);\n+extern int decompress_into(ext_stream *stream, unsigned char *out, unsigned long out_size);\n \n-extern int decompress_next(z_stream *stream, int flush);\n-extern int decompress_next_from(z_stream *stream, unsigned char *in, unsigned long in_size, int flush);\n-extern int decompress_next_into(z_stream *stream, unsigned char *out, unsigned long out_size, int flush);\n+extern int decompress_next(ext_stream *stream, int flush);\n+extern int decompress_next_from(ext_stream *stream, unsigned char *in, unsigned long in_size, int flush);\n+extern int decompress_next_into(ext_stream *stream, unsigned char *out, unsigned long out_size, int flush);\n \n-extern unsigned long decompress_free(z_stream *stream);\n+extern unsigned long decompress_free(ext_stream *stream);\n \n extern unsigned long decompress_all(unsigned char *in, unsigned long in_size,\n                                     unsigned char *out, unsigned long out_size);\n-- \n1.5.4.rc2.98.g58cd2\n"},{"id":"65239","messageId":"1200230678-18188-2-git-send-email-mcostalba@gmail.com","threadId":"11604","inReplyTo":"1200230678-18188-1-git-send-email-mcostalba@gmail.com","subject":"[PATCH 2/3] Rename z_stream to ext_stream across the sources","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-01-13T13:24:37Z","receivedAt":"2008-01-13T13:24:37Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Also fix the places where z_stream member access\nis open coded.\n\nIn the future we could hide direct z_stream handling\nbehind some helpers, but for now leave it like it is\nnow to better understand what is going on.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n builtin-pack-objects.c   |    8 +++---\n builtin-unpack-objects.c |    6 ++--\n http-push.c              |   12 +++++-----\n http-walker.c            |    6 ++--\n index-pack.c             |    6 ++--\n sha1_file.c              |   56 +++++++++++++++++++++++-----------------------\n 6 files changed, 47 insertions(+), 47 deletions(-)\n\ndiff --git a/builtin-pack-objects.c b/builtin-pack-objects.c\nindex d2865fe..5165a23 100644\n--- a/builtin-pack-objects.c\n+++ b/builtin-pack-objects.c\n@@ -300,7 +300,7 @@ static int check_pack_inflate(struct packed_git *p,\n \t\toff_t len,\n \t\tunsigned long expect)\n {\n-\tz_stream stream;\n+\text_stream stream;\n \tunsigned char fakebuf[4096], *in;\n \tunsigned int in_size = 0;\n \tint st;\n@@ -310,12 +310,12 @@ static int check_pack_inflate(struct packed_git *p,\n \t\tdecompress_into(&stream, fakebuf, sizeof(fakebuf));\n \t\tin = use_pack(p, w_curs, offset, &in_size);\n \t\tst = decompress_next_from(&stream, in, in_size, Z_FINISH);\n-\t\toffset += stream.next_in - in;\n+\t\toffset += stream.z.next_in - in;\n \t} while (st == Z_OK || st == Z_BUF_ERROR);\n \tdecompress_free(&stream);\n \treturn (st == Z_STREAM_END &&\n-\t\tstream.total_out == expect &&\n-\t\tstream.total_in == len) ? 0 : -1;\n+\t\tstream.z.total_out == expect &&\n+\t\tstream.z.total_in == len) ? 0 : -1;\n }\n \n static int check_pack_crc(struct packed_git *p, struct pack_window **w_curs,\ndiff --git a/builtin-unpack-objects.c b/builtin-unpack-objects.c\nindex c996560..066bf06 100644\n--- a/builtin-unpack-objects.c\n+++ b/builtin-unpack-objects.c\n@@ -61,7 +61,7 @@ static void use(int bytes)\n \n static void *get_data(unsigned long size)\n {\n-\tz_stream stream;\n+\text_stream stream;\n \tunsigned char *buf = xmalloc(size);;\n \n \tdecompress_alloc(&stream);\n@@ -71,8 +71,8 @@ static void *get_data(unsigned long size)\n \t\t/* fill() modifies len, so be sure is evaluated as first */\n \t\tvoid* tmp = fill(1);\n \t\tint ret = decompress_next_from(&stream, tmp, len, Z_NO_FLUSH);\n-\t\tuse(len - stream.avail_in);\n-\t\tif (stream.total_out == size && ret == Z_STREAM_END)\n+\t\tuse(len - stream.z.avail_in);\n+\t\tif (stream.z.total_out == size && ret == Z_STREAM_END)\n \t\t\tbreak;\n \t\tif (ret != Z_OK) {\n \t\t\terror(\"decompress returned %d\\n\", ret);\ndiff --git a/http-push.c b/http-push.c\nindex ec0568c..c7ea871 100644\n--- a/http-push.c\n+++ b/http-push.c\n@@ -127,7 +127,7 @@ struct transfer_request\n \tlong http_code;\n \tunsigned char real_sha1[20];\n \tSHA_CTX c;\n-\tz_stream stream;\n+\text_stream stream;\n \tint zret;\n \tint rename;\n \tvoid *userData;\n@@ -209,8 +209,8 @@ static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb,\n \t\trequest->zret = decompress_next_into(&request->stream, expn,\n \t\t                                     sizeof(expn), Z_SYNC_FLUSH);\n \t\tSHA1_Update(&request->c, expn,\n-\t\t\t    sizeof(expn) - request->stream.avail_out);\n-\t} while (request->stream.avail_in && request->zret == Z_OK);\n+\t\t\t    sizeof(expn) - request->stream.z.avail_out);\n+\t} while (request->stream.z.avail_in && request->zret == Z_OK);\n \tdata_received++;\n \treturn size;\n }\n@@ -483,7 +483,7 @@ static void start_put(struct transfer_request *request)\n \tunsigned long len;\n \tint hdrlen;\n \tssize_t size;\n-\tz_stream stream;\n+\text_stream stream;\n \n \tunpacked = read_sha1_file(request->obj->sha1, &type, &len);\n \thdrlen = sprintf(hdr, \"%s %lu\", typename(type), len) + 1;\n@@ -501,8 +501,8 @@ static void start_put(struct transfer_request *request)\n \tcompress_next(&stream, Z_NO_FLUSH);\n \n \t/* Then the data itself.. */\n-\tstream.next_in = unpacked;\n-\tstream.avail_in = len;\n+\tstream.z.next_in = unpacked;\n+\tstream.z.avail_in = len;\n \tcompress_next(&stream, Z_FINISH);\n \n \trequest->buffer.buf.len = compress_free(&stream);\ndiff --git a/http-walker.c b/http-walker.c\nindex b1d2a28..1581746 100644\n--- a/http-walker.c\n+++ b/http-walker.c\n@@ -38,7 +38,7 @@ struct object_request\n \tlong http_code;\n \tunsigned char real_sha1[20];\n \tSHA_CTX c;\n-\tz_stream stream;\n+\text_stream stream;\n \tint zret;\n \tint rename;\n \tstruct active_request_slot *slot;\n@@ -83,8 +83,8 @@ static size_t fwrite_sha1_file(void *ptr, size_t eltsize, size_t nmemb,\n \t\tobj_req->zret = decompress_next_into(&obj_req->stream, expn,\n                                              sizeof(expn), Z_SYNC_FLUSH);\n \t\tSHA1_Update(&obj_req->c, expn,\n-\t\t\t    sizeof(expn) - obj_req->stream.avail_out);\n-\t} while (obj_req->stream.avail_in && obj_req->zret == Z_OK);\n+\t\t\t    sizeof(expn) - obj_req->stream.z.avail_out);\n+\t} while (obj_req->stream.z.avail_in && obj_req->zret == Z_OK);\n \tdata_received++;\n \treturn size;\n }\ndiff --git a/index-pack.c b/index-pack.c\nindex 929de39..7881a0b 100644\n--- a/index-pack.c\n+++ b/index-pack.c\n@@ -166,7 +166,7 @@ static void bad_object(unsigned long offset, const char *format, ...)\n \n static void *unpack_entry_data(unsigned long offset, unsigned long size)\n {\n-\tz_stream stream;\n+\text_stream stream;\n \tvoid *buf = xmalloc(size);\n \n \tdecompress_alloc(&stream);\n@@ -176,8 +176,8 @@ static void *unpack_entry_data(unsigned long offset, unsigned long size)\n \t\t/* fill() modifies len, so be sure is evaluated as first */\n \t\tvoid* tmp = fill(1);\n \t\tint ret = decompress_next_from(&stream, tmp, input_len, Z_NO_FLUSH);\n-\t\tuse(input_len - stream.avail_in);\n-\t\tif (stream.total_out == size && ret == Z_STREAM_END)\n+\t\tuse(input_len - stream.z.avail_in);\n+\t\tif (stream.z.total_out == size && ret == Z_STREAM_END)\n \t\t\tbreak;\n \t\tif (ret != Z_OK)\n \t\t\tbad_object(offset, \"decompress returned %d\", ret);\ndiff --git a/sha1_file.c b/sha1_file.c\nindex 708727a..a978f13 100644\n--- a/sha1_file.c\n+++ b/sha1_file.c\n@@ -1052,7 +1052,7 @@ unsigned long unpack_object_header_gently(const unsigned char *buf, unsigned lon\n \treturn used;\n }\n \n-static int unpack_sha1_header(z_stream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz)\n+static int unpack_sha1_header(ext_stream *stream, unsigned char *map, unsigned long mapsize, void *buffer, unsigned long bufsiz)\n {\n \tunsigned long size, used;\n \tstatic const char valid_loose_object_type[8] = {\n@@ -1087,19 +1087,19 @@ static int unpack_sha1_header(z_stream *stream, unsigned char *map, unsigned lon\n \tdecompress_from(stream, map, mapsize);\n \n \t/* And generate the fake traditional header */\n-\tstream->total_out = 1 + snprintf(buffer, bufsiz, \"%s %lu\",\n+\tstream->z.total_out = 1 + snprintf(buffer, bufsiz, \"%s %lu\",\n \t\t\t\t\t typename(type), size);\n \treturn 0;\n }\n \n-static void *unpack_sha1_rest(z_stream *stream, void *buffer, unsigned long size, const unsigned char *sha1)\n+static void *unpack_sha1_rest(ext_stream *stream, void *buffer, unsigned long size, const unsigned char *sha1)\n {\n \tint bytes = strlen(buffer) + 1;\n \tunsigned char *buf = xmalloc(1+size);\n \tunsigned long n;\n \tint status = Z_OK;\n \n-\tn = stream->total_out - bytes;\n+\tn = stream->z.total_out - bytes;\n \tif (n > size)\n \t\tn = size;\n \tmemcpy(buf, (char *) buffer + bytes, n);\n@@ -1123,14 +1123,14 @@ static void *unpack_sha1_rest(z_stream *stream, void *buffer, unsigned long size\n \t\t\tstatus = decompress_next(stream, Z_FINISH);\n \t}\n \tbuf[size] = 0;\n-\tif (status == Z_STREAM_END && !stream->avail_in) {\n+\tif (status == Z_STREAM_END && !stream->z.avail_in) {\n \t\tdecompress_free(stream);\n \t\treturn buf;\n \t}\n \n \tif (status < 0)\n \t\terror(\"corrupt loose object '%s'\", sha1_to_hex(sha1));\n-\telse if (stream->avail_in)\n+\telse if (stream->z.avail_in)\n \t\terror(\"garbage at end of loose object '%s'\",\n \t\t      sha1_to_hex(sha1));\n \tfree(buf);\n@@ -1191,7 +1191,7 @@ static int parse_sha1_header(const char *hdr, unsigned long *sizep)\n static void *unpack_sha1_file(void *map, unsigned long mapsize, enum object_type *type, unsigned long *size, const unsigned char *sha1)\n {\n \tint ret;\n-\tz_stream stream;\n+\text_stream stream;\n \tchar hdr[8192];\n \n \tret = unpack_sha1_header(&stream, map, mapsize, hdr, sizeof(hdr));\n@@ -1207,7 +1207,7 @@ unsigned long get_size_from_delta(struct packed_git *p,\n {\n \tconst unsigned char *data;\n \tunsigned char delta_head[20], *in;\n-\tz_stream stream;\n+\text_stream stream;\n \tint st;\n \tunsigned int in_size = 0;\n \n@@ -1217,11 +1217,11 @@ unsigned long get_size_from_delta(struct packed_git *p,\n \tdo {\n \t\tin = use_pack(p, w_curs, curpos, &in_size);\n \t\tst = decompress_next_from(&stream, in, in_size, Z_FINISH);\n-\t\tcurpos += stream.next_in - in;\n+\t\tcurpos += stream.z.next_in - in;\n \t} while ((st == Z_OK || st == Z_BUF_ERROR) &&\n-\t\t stream.total_out < sizeof(delta_head));\n+\t\t stream.z.total_out < sizeof(delta_head));\n \tdecompress_free(&stream);\n-\tif ((st != Z_STREAM_END) && stream.total_out != sizeof(delta_head))\n+\tif ((st != Z_STREAM_END) && stream.z.total_out != sizeof(delta_head))\n \t\tdie(\"delta data unpack-initial failed\");\n \n \t/* Examine the initial part of the delta to figure out\n@@ -1416,7 +1416,7 @@ static void *unpack_compressed_entry(struct packed_git *p,\n \t\t\t\t    unsigned long size)\n {\n \tint st;\n-\tz_stream stream;\n+\text_stream stream;\n \tunsigned char *buffer, *in;\n \tunsigned int in_size = 0;\n \n@@ -1427,10 +1427,10 @@ static void *unpack_compressed_entry(struct packed_git *p,\n \tdo {\n \t\tin = use_pack(p, w_curs, curpos, &in_size);\n \t\tst = decompress_next_from(&stream, in, in_size, Z_FINISH);\n-\t\tcurpos += stream.next_in - in;\n+\t\tcurpos += stream.z.next_in - in;\n \t} while (st == Z_OK || st == Z_BUF_ERROR);\n \tdecompress_free(&stream);\n-\tif ((st != Z_STREAM_END) || stream.total_out != size) {\n+\tif ((st != Z_STREAM_END) || stream.z.total_out != size) {\n \t\tfree(buffer);\n \t\treturn NULL;\n \t}\n@@ -1762,7 +1762,7 @@ static int sha1_loose_object_info(const unsigned char *sha1, unsigned long *size\n \tint status;\n \tunsigned long mapsize, size;\n \tvoid *map;\n-\tz_stream stream;\n+\text_stream stream;\n \tchar hdr[32];\n \n \tmap = map_sha1_file(sha1, &mapsize);\n@@ -2033,7 +2033,7 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha\n {\n \tint size, ret;\n \tunsigned char *compressed;\n-\tz_stream stream;\n+\text_stream stream;\n \tunsigned char sha1[20];\n \tchar *filename;\n \tstatic char tmpfile[PATH_MAX];\n@@ -2084,8 +2084,8 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha\n \tcompress_next(&stream, Z_NO_FLUSH);\n \n \t/* Then the data itself.. */\n-\tstream.next_in = buf;\n-\tstream.avail_in = len;\n+\tstream.z.next_in = buf;\n+\tstream.z.avail_in = len;\n \tret = compress_next(&stream, Z_FINISH);\n \tif (ret != Z_STREAM_END)\n \t\tdie(\"unable to deflate new object %s (%d)\", sha1_to_hex(sha1), ret);\n@@ -2109,7 +2109,7 @@ int write_sha1_file(void *buf, unsigned long len, const char *type, unsigned cha\n static void *repack_object(const unsigned char *sha1, unsigned long *objsize)\n {\n \tsize_t size;\n-\tz_stream stream;\n+\text_stream stream;\n \tunsigned char *unpacked;\n \tunsigned long len;\n \tenum object_type type;\n@@ -2135,8 +2135,8 @@ static void *repack_object(const unsigned char *sha1, unsigned long *objsize)\n \tcompress_next(&stream, Z_NO_FLUSH);\n \n \t/* Then the data itself.. */\n-\tstream.next_in = unpacked;\n-\tstream.avail_in = len;\n+\tstream.z.next_in = unpacked;\n+\tstream.z.avail_in = len;\n \tcompress_next(&stream, Z_FINISH);\n \n \t*objsize = compress_free(&stream);\n@@ -2167,7 +2167,7 @@ int write_sha1_from_fd(const unsigned char *sha1, int fd, char *buffer,\n {\n \tchar tmpfile[PATH_MAX];\n \tint local;\n-\tz_stream stream;\n+\text_stream stream;\n \tunsigned char real_sha1[20];\n \tunsigned char discard[4096];\n \tint ret;\n@@ -2193,13 +2193,13 @@ int write_sha1_from_fd(const unsigned char *sha1, int fd, char *buffer,\n \t\t\tdo {\n \t\t\t\tret = decompress_next_into(&stream, discard, sizeof(discard), Z_SYNC_FLUSH);\n \t\t\t\tSHA1_Update(&c, discard, sizeof(discard) -\n-\t\t\t\t\t    stream.avail_out);\n-\t\t\t} while (stream.avail_in && ret == Z_OK);\n-\t\t\tif (write_buffer(local, buffer, *bufposn - stream.avail_in) < 0)\n+\t\t\t\t\t    stream.z.avail_out);\n+\t\t\t} while (stream.z.avail_in && ret == Z_OK);\n+\t\t\tif (write_buffer(local, buffer, *bufposn - stream.z.avail_in) < 0)\n \t\t\t\tdie(\"unable to write sha1 file\");\n-\t\t\tmemmove(buffer, buffer + *bufposn - stream.avail_in,\n-\t\t\t\tstream.avail_in);\n-\t\t\t*bufposn = stream.avail_in;\n+\t\t\tmemmove(buffer, buffer + *bufposn - stream.z.avail_in,\n+\t\t\t\tstream.z.avail_in);\n+\t\t\t*bufposn = stream.z.avail_in;\n \t\t\tif (ret != Z_OK)\n \t\t\t\tbreak;\n \t\t}\n-- \n1.5.4.rc2.98.g58cd2\n"},{"id":"65240","messageId":"1200230678-18188-3-git-send-email-mcostalba@gmail.com","threadId":"11604","inReplyTo":"1200230678-18188-2-git-send-email-mcostalba@gmail.com","subject":"[PATCH 3/3] Abstract out zlib","fromName":"Marco Costalba","fromEmail":"mcostalba@gmail.com","sentAt":"2008-01-13T13:24:38Z","receivedAt":"2008-01-13T13:24:38Z","isPatch":true,"sender":{"key":"mcostalba@gmail.com","avatar":null},"body":"Finally remove the hardcoded link between\ncompression helpers and zlib.\n\nAssociation bewteen a stream and the compression library\nis done at stream init time through a set of function\npointers. Library choice is based on compression level\nparameter passed to compress_alloc(), this allow us to\nstay 100% back compatible with current code.\n\nPatch also adds the the necessary zlib engine plugin,\nthis turns out to be trivial since we have modeled\neverything around zlib.\n\nSigned-off-by: Marco Costalba <mcostalba@gmail.com>\n---\n compress.c |   55 ++++++++++++++++++++++++++++++++++++++++++++++---------\n compress.h |    6 ++++++\n 2 files changed, 52 insertions(+), 9 deletions(-)\n\ndiff --git a/compress.c b/compress.c\nindex b28c389..09a5df0 100644\n--- a/compress.c\n+++ b/compress.c\n@@ -1,6 +1,41 @@\n #include \"cache.h\"\n #include \"compress.h\"\n \n+/* Default zlib engine plugin definition\n+ *\n+ * Because everything is modeled after zlib the\n+ * corresponding engine wrappers here are trivial\n+ */\n+static int zlib_deflateInit(z_stream *s, int l) { return deflateInit(s, l); }\n+static int zlib_deflate(z_stream *s, int f) { return deflate(s, f); }\n+static int zlib_deflateEnd(z_stream *s) { return deflateEnd(s); }\n+static unsigned long zlib_deflateBound(z_stream *s, unsigned long sz) { return deflateBound(s, sz); }\n+\n+static int zlib_inflateInit(z_stream *s) { return inflateInit(s); }\n+static int zlib_inflate(z_stream *s, int f) { return inflate(s, f); }\n+static int zlib_inflateEnd(z_stream *s) { return inflateEnd(s); }\n+\n+\n+/* link the stream to the compression library functions */\n+static int register_engine(ext_stream *stream, int level)\n+{\n+\tswitch (level) {\n+\tcase LZO_COMPRESSION:\n+\t\tdie (\"LZO compression still not implemented\");\n+\t\tbreak;\n+\tdefault: /* assumed to be zlib */\n+\t\tstream->deflateInit_fn  = zlib_deflateInit;\n+\t\tstream->deflate_fn      = zlib_deflate;\n+\t\tstream->deflateEnd_fn   = zlib_deflateEnd;\n+\t\tstream->deflateBound_fn = zlib_deflateBound;\n+\t\tstream->inflateInit_fn  = zlib_inflateInit;\n+\t\tstream->inflate_fn      = zlib_inflate;\n+\t\tstream->inflateEnd_fn   = zlib_inflateEnd;\n+\t\tbreak;\n+\t};\n+\treturn Z_OK;\n+}\n+\n /*\n  *     Compression helpers\n  */\n@@ -8,8 +43,9 @@\n unsigned long compress_alloc(ext_stream *stream, int level, unsigned long size)\n {\n \tmemset(stream, 0, sizeof(*stream));\n-\tdeflateInit(&stream->z, level);\n-\treturn deflateBound(&stream->z, size);\n+\tregister_engine(stream, level);\n+\tstream->deflateInit_fn(&stream->z, level);\n+\treturn stream->deflateBound_fn(&stream->z, size);\n }\n \n int compress_start(ext_stream *stream,\n@@ -28,7 +64,7 @@ int compress_next(ext_stream *stream, int flush)\n \tint result;\n \n \tdo {\n-\t\tresult = deflate(&stream->z, flush);\n+\t\tresult = stream->deflate_fn(&stream->z, flush);\n \t} while (result == Z_OK);\n \n \treturn result;\n@@ -36,7 +72,7 @@ int compress_next(ext_stream *stream, int flush)\n \n unsigned long compress_free(ext_stream *stream)\n {\n-\tdeflateEnd(&stream->z);\n+\tstream->deflateEnd_fn(&stream->z);\n \treturn stream->z.total_out;\n }\n \n@@ -68,7 +104,8 @@ unsigned long compress_all(int level, unsigned char *in,\n int decompress_alloc(ext_stream *stream)\n {\n \tmemset(stream, 0, sizeof(*stream));\n-\treturn inflateInit(&stream->z);\n+\tregister_engine(stream, Z_DEFAULT_COMPRESSION); // FIXME for now zlib assumed\n+\treturn stream->inflateInit_fn(&stream->z);\n }\n \n int decompress_from(ext_stream *stream, unsigned char *in, unsigned long in_size)\n@@ -87,24 +124,24 @@ int decompress_into(ext_stream *stream, unsigned char *out, unsigned long out_si\n \n int decompress_next(ext_stream *stream, int flush)\n {\n-\treturn inflate(&stream->z, flush);\n+\treturn stream->inflate_fn(&stream->z, flush);\n }\n \n int decompress_next_from(ext_stream *stream, unsigned char *in, unsigned long in_size, int flush)\n {\n \tdecompress_from(stream, in, in_size);\n-\treturn inflate(&stream->z, flush);\n+\treturn stream->inflate_fn(&stream->z, flush);\n }\n \n int decompress_next_into(ext_stream *stream, unsigned char *out, unsigned long out_size, int flush)\n {\n \tdecompress_into(stream, out, out_size);\n-\treturn inflate(&stream->z, flush);\n+\treturn stream->inflate_fn(&stream->z, flush);\n }\n \n unsigned long decompress_free(ext_stream *stream)\n {\n-\tinflateEnd(&stream->z);\n+\tstream->inflateEnd_fn(&stream->z);\n \treturn stream->z.total_out;\n }\n \ndiff --git a/compress.h b/compress.h\nindex d1de31f..151234a 100644\n--- a/compress.h\n+++ b/compress.h\n@@ -1,6 +1,12 @@\n #ifndef COMPRESS_H\n #define COMPRESS_H\n \n+/* Add here custom compression levels. First 0-9\n+ * and -1 are reserved values used by zlib\n+ */\n+#define LZO_COMPRESSION 99\n+\n+\n /* Any compress/decompress engine must implement all the\n  * below functions that are modeled after the zlib ones.\n  */\n-- \n1.5.4.rc2.98.g58cd2\n"}]}