Re: Compression speed for large files

From: Joachim Berdal Haga <cjhaga@student.matnat.uio.no>
Date: Jul 3, 2006, 13:32 UTC
Message-ID: <44A91C7A.6090902@fys.uio.no>
In-Reply-To: <81b0412b0607030503p63b4ee31v7776bd155d3dab29@mail.gmail.com>

Alex Riesen wrote:

Show 8 quoted lines

> On 7/3/06, Joachim B Haga <cjhaga@fys.uio.no> wrote:
>> So: is it a good idea to change to faster compression, at least for 
>> larger files? From my (limited) testing I would suggest using 
>> Z_BEST_COMPRESSION only for small files (perhaps <1MB?) and 
>> Z_DEFAULT_COMPRESSION/Z_BEST_SPEED for
>> larger ones.
> 
> Probably yes, as a per-repo config option.

I can send a patch later. If it's to be a per-repo option, it's probably too confusing with several values. Is it ok with

core.compression = [-1..9]

where the numbers are the zlib/gzip constants,
   -1 = zlib default (currently 6)
    0 = no compression
1..9 = various speed/size tradeoffs (9 is git default)

Btw; I just tested the kernel sources. With gzip only, but files 
compressed individually:
   time find . -type f | xargs gzip -9 -c | wc -c

I found the space saving from -6 to -9 to be under 0.6%, at double the CPU time. So perhaps Z_DEFAULT_COMPRESSION would be good as default.

-j