Re: Decompression speed: zip vs lzo

From: Marco Costalba <mcostalba@gmail.com>
Date: Jan 10, 2008, 06:55 UTC
Message-ID: <e5bfff550801092255wc852252m9086567a88b1ae99@mail.gmail.com>
In-Reply-To: <alpine.LFD.1.00.0801092234130.3054@xanadu.home>

On Jan 10, 2008 4:41 AM, Nicolas Pitre <nico@cam.org> wrote:

Show 11 quoted lines

> On Wed, 9 Jan 2008, Johannes Schindelin wrote:
>
> > I agree that gzip is already fast enough.
> >
> > However, pack v4 had more goodies than just being faster; it also promised
> > to have smaller packs.
>
> Right, like not having to compress tree objects and half of commit
> objects at all.
>
>

Decompression speed has been shown to be a bottle neck on some tests involving mainly 'git log'.

Regarding back compatibility I really don't know at what level git functions actually need to know the compression format, looking at the code I would say at very low level, functions that deal directly with inflate() and friends are few [1] and not directly connected to UI, nor to git config. Is this compression format something user should know/care? and if yes why?

In my tests the assumption of a source files tar ball is unrealistic, to test the final size difference I would like testing different compressions on a big already packaged but still not zipped file. Someone could be so kind to hint me on how to create such a package with good quality, i.e. with packaging levels similar to what is done for public repos?

This does not realistically tests speed because as Junio pointed out the real decompressing schema is different: many calls on small objects, not one call on a big one. But if final size is acceptable we can go on more difficult tests.

Marco

[1] where inflate() is called:

-inflate_it() in builtin-apply.c -check_pack_inflate() in builtin-pack-objects.c -get_data() in builtin-unpack-objects.c -fwrite_sha1_file() in http-push.c and http-walker.c [mmm interesting same function in two files, also the signature and the contents seems the same....] -unpack_entry_data() in index-pack.c -unpack_sha1_header(), unpack_sha1_rest(), get_size_from_delta(), unpack_compressed_entry, write_sha1_from_fd() in sha1_file.c