git/list[1] front-page[2] threads[3] people[4] search[5] about
 

pack-object poor performance (with large number of objects?)

From
Piotr Krukowiecki <piotr.krukowiecki@gmail.com>
Date
Oct 3, 2011, 14:43 UTC
Message-ID
<CAA01CspZijOO_xbR=OcaRaesTeSy=6RM4DR01-07qimVzxvJZA@mail.gmail.com>
Hi,

I'm having poor git gc (pack-object) performance. Please read below for details. What can I do to improve the performance/debug the reason for the slowness? Should I leave the process running over night, or should I stop it (for debugging)? CCing people who posted some patches/benchmarks for pack-objects recently.

git gc was first run automatically by git svn clone. It found 1544673 objects and worked for 50 minutes until I've killed it.

Then I've run it by hand with --aggresive (because I've found on Internet it helped in some cases). It found 1742200 objects this time. At this moment it's been working for about 90 minutes.

The large number of unpacked objects is probably caused by me - I've disabled auto gc when I was cloning from svn (I though it might speed up things if it didn't repack several times during clone, only once afterwards).

My git version is 1.7.7.rc3.4.g8d714 The file system is ext4.

First run process tree:
pkruk    27873  0.0  0.0  15704   816 pts/2    S+   11:53   0:00
       |           |                       \_ git gc --auto
pkruk    27885  0.0  0.0  15704   776 pts/2    S+   11:53   0:00
       |           |                           \_ git repack -d -l
pkruk    27886  0.0  0.0   4220   608 pts/2    S+   11:53   0:00
       |           |                               \_ /bin/sh
/usr/local/stow/git-master/libexec/git-core/git-repack -d -l
pkruk    27897  3.6  9.3 1136072 381148 pts/2  D+   11:53   5:51
       |           |                                   \_ git
pack-objects --keep-true-parents --honor-pack-keep --non-empty --all
--reflog --unpacked --incremental --local --delta-base-offset
/home/pkruk/dv/devel1_git_repos/.git/objects/pack/.tmp-27886-pack
Second run process tree:
pkruk     6171  0.0  0.0  15704  1428 pts/2    S+   14:34   0:00
       |           |               \_ git gc --aggressive
pkruk     6174  0.0  0.0  15704  1356 pts/2    S+   14:34   0:00
       |           |                   \_ git repack -d -l -f
--depth=250 --window=250 -A
pkruk     6175  0.0  0.0   4220   648 pts/2    S+   14:34   0:00
       |           |                       \_ /bin/sh
/usr/local/stow/git-master/libexec/git-core/git-repack -d -l -f
--depth=250 --window=250 -A
pkruk     6189  4.9 10.5 1143640 427396 pts/2  D+   14:34   4:50
       |           |                           \_ git pack-objects
--keep-true-parents --honor-pack-keep --non-empty --all --reflog
--unpack-unreachable --local --no-reuse-delta --depth=250 --window=250
--delta-base-offset
/home/pkruk/dv/devel1_git_repos/.git/objects/pack/.tmp-6175-pack
-- 
Piotr Krukowiecki
Next: Shawn Pearce
Message 1 of 11 in “pack-object poor performance (with large number of objects?)”
  1. Piotr KrukowieckiOct 3, 2011
  2. Shawn PearceOct 3, 2011
  3. Piotr KrukowieckiOct 3, 2011
  4. Junio C HamanoOct 3, 2011
  5. Piotr KrukowieckiOct 4, 2011
  6. Jeff KingOct 4, 2011
  7. Piotr KrukowieckiOct 4, 2011
  8. Jeff KingOct 4, 2011
  9. Piotr KrukowieckiOct 4, 2011
  10. Jeff KingOct 4, 2011
  11. Piotr KrukowieckiOct 5, 2011

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.