git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Performance of various compressors

From
ALAaron Lehmann <aaronl@vitelus.com>
Date
Apr 22, 2005, 20:38 UTC
Message-ID
<20050422203801.GE7437@vitelus.com>
In-Reply-To
<426734DE.3040606@timesys.com>
On Wed, Apr 20, 2005 at 10:06:38PM -0700, Mike Taht wrote:
> That doing the compression at a level of 3, rather than the max of 9, 
> cuts the cpu time required for a big git commit by over half, and that 
> that actually translates into a win on the I/O to disk. (these tests 
> were performed on a dual opteron 842)

If (de)compression is slowing things down, you might want to check out lzo (http://www.oberhumer.com/opensource/lzo/). I tested it on the 2.6.11 kernel source and found that lzo -7 output is only 2% larger than gzip -3, but lzo decompression is almost 3 times faster. The downside is that lzo took 5 times longer to perform the compression at -7. Compression with lzo -3 is 3.5 times faster than gzip -3, but it produces a file that's 37% bigger. Unfortunately, lzo has no settings in between -3 and -7. I'd expect git to be more sensitive to decompression speeds, though.

BTW, lzo decompression speed is not affected by the compression level.
Previous: H. Peter AnvinNext: Klaus Robert Suetterlin
Message 10 of 11 in “Performance of various compressors”
  1. Mike TahtApr 21, 2005
  2. Mike TahtApr 21, 2005
  3. experimental - Performance of various compressorsMike Taht, Apr 21, 2005
  4. HOWTO: PATCH: don't hardcode path-to-bash, use sys/limits.hKlaus Robert Suetterlin, Apr 21, 2005
  5. Alecs KingApr 21, 2005
  6. #!/bin/sh --> #!/usr/bin/env bashAlecs King, Apr 21, 2005
  7. H. Peter AnvinApr 22, 2005
  8. David A. WheelerApr 23, 2005
  9. H. Peter AnvinApr 23, 2005
  10. Aaron LehmannApr 22, 2005
  11. git I/O performance (was: Performance of various compressors)Klaus Robert Suetterlin, Apr 25, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.