git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Performance of various compressors

From
MTMike Taht <mike.taht@timesys.com>
Date
Apr 21, 2005, 05:06 UTC
Message-ID
<426734DE.3040606@timesys.com>

I started rolling a tool to measure various aspects of git performance. I will start looking at merge next, and at workloads different from the kernel (gcc4 anyone?) ...

The only data points worth sharing a this point are:

That doing the compression at a level of 3, rather than the max of 9, cuts the cpu time required for a big git commit by over half, and that that actually translates into a win on the I/O to disk. (these tests were performed on a dual opteron 842)

The benefits of compression aren't very much for git right now.
And: A big git commit is I/O bound. But we knew that. Maybe it's 
possible to make it less I/O bound.
Git branch: 7a4c67965de68ae7bc7aa1fde33f8eb9d8114697
Tree: 2.6.11.7 source tree
Branch: N/a
Merge File: N/a
HW: dual opteron 242
Mem: 1GB
Disk: seagate barracuda
Filesystem: Reiser3
Git add: N/a
Cache: Hot
Git Commit: 44.97user 5.94system 1:45.24elapsed 48%CPU
Git Merge:
Options:
Feature: Test of compression=9 (std git)

du -s .git/objects 110106 # du is probably not the right thing du -s --apparent-size .git/objects 58979

Git branch: 9e272677621c91784cf2533123a41745178f0701
Tree: 2.6.11.7 source tree
Branch: N/a
Merge File: N/a
HW: dual opteron 242
Mem: 1GB
Disk: seagate barracuda
Disk mode: udma5
Filesystem: Reiser3
Git add: N/a
Cache: Hot
Git Commit: 16.79user 6.15system 1:21.92elapsed 28%CPU
Git Merge:
Options:
Feature: Test of compression=3 (std git)

du -s .git/objects 115218 du -s --apparent-size .git/objects 64274

There's some variety in the best/worst case timings for I/O for the compressor=3 case...

16.79user 6.15system 1:21.92elapsed 28%CPU 16.68user 5.71system 1:13.19elapsed 30%CPU

-- 
Mike Taht


lastly -

Timings of git commit with tmpfs (note, these were done with an ancient, 
5 hour old version of git and the script)

Hot cache, tmpfs .git compression=9
44.97user 2.76system 0:47.72elapsed 100%CPU

Hot cache, tmpfs .git, compression=6
Wed Apr 20 20:18:11 PDT 2005
23.55user 2.83system 0:26.36elapsed 100%CPU (0avgtext+0avgdata 
0maxresident)k
0inputs+0outputs (0major+568680minor)pagefaults 0swaps
109620  .git/objects
58618   .git/objects
Next: Mike Taht
Message 1 of 11 in “Performance of various compressors”
  1. Mike TahtApr 21, 2005
  2. Mike TahtApr 21, 2005
  3. experimental - Performance of various compressorsMike Taht, Apr 21, 2005
  4. HOWTO: PATCH: don't hardcode path-to-bash, use sys/limits.hKlaus Robert Suetterlin, Apr 21, 2005
  5. Alecs KingApr 21, 2005
  6. #!/bin/sh --> #!/usr/bin/env bashAlecs King, Apr 21, 2005
  7. H. Peter AnvinApr 22, 2005
  8. David A. WheelerApr 23, 2005
  9. H. Peter AnvinApr 23, 2005
  10. Aaron LehmannApr 22, 2005
  11. git I/O performance (was: Performance of various compressors)Klaus Robert Suetterlin, Apr 25, 2005

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.