'git gc --aggressive' effectively unusable
- From
- Frans Pop <elendil@planet.nl>
- Date
- Apr 2, 2010, 22:05 UTC
- Message-ID
- <201004030005.35737.elendil@planet.nl>
Note: this is on a different repo from the 'git reflog expire --all' I reported a bit earlier.
I have a git-svn checkout of a subversion repo which I wanted to compress as much as possible. 'git gc --aggressive' starts to run fairly well, but eats more and more memory and gets slower and slower. After it gets to about 45% or 50% progress slows down noticeably and so far I haven't had the patience to let it finish (40 minutes is already way too long).
A regular 'git gc' run completes without any problems.
$ du -sh .git/ 612M .git/
Special about this repo is that it contains two huge objects [1], which
could maybe be a factor:
size pack SHA
- packages/po/sublevel4/da.po:
495661 4654 801cd6451ece536c0ab41f79e09fc52efdf3361f
- packages/arch/powerpc/quik-installer/debian/po/da.po
149515 1403 83a787b20817dc4d72db052de4055e7a7c9221d7 Below some output from top and of the progress of the command showing the problem. Check the change in number of compressed objects against the timestamps from top.
Cheers, FJP
[1] Caused by a bug in a script a couple of years back.
$ git gc --aggressive
Counting objects: 843342, done. Delta compression using up to 2 threads. Compressing objects: 53% (449663/836424)
top - 22:55:02 up 18 min, 1 user, load average: 1.83, 1.68, 1.07 Tasks: 161 total, 1 running, 160 sleeping, 0 stopped, 0 zombie Cpu0 : 91.4%us, 0.7%sy, 0.0%ni, 1.3%id, 6.6%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 : 97.7%us, 0.3%sy, 0.0%ni, 1.3%id, 0.7%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 2034284k total, 2018288k used, 15996k free, 10188k buffers Swap: 2097148k total, 22612k used, 2074536k free, 449444k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5861 fjp 20 0 1775m 1.3g 194m S 188 66.7 21:10.89 git
Counting objects: 843342, done. Delta compression using up to 2 threads. Compressing objects: 58% (486001/836424)
top - 23:00:12 up 23 min, 1 user, load average: 1.96, 1.84, 1.30 Tasks: 158 total, 2 running, 156 sleeping, 0 stopped, 0 zombie Cpu0 : 98.3%us, 0.7%sy, 0.0%ni, 0.7%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 : 87.4%us, 1.7%sy, 0.0%ni, 0.0%id, 10.6%wa, 0.0%hi, 0.3%si, 0.0%st Mem: 2034284k total, 2017516k used, 16768k free, 4696k buffers Swap: 2097148k total, 22572k used, 2074576k free, 336944k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5861 fjp 20 0 1903m 1.4g 172m S 182 71.4 30:37.58 git
Counting objects: 843342, done. Delta compression using up to 2 threads. Compressing objects: 61% (515958/836424)
top - 23:05:56 up 29 min, 1 user, load average: 1.68, 1.85, 1.48 Tasks: 159 total, 1 running, 158 sleeping, 0 stopped, 0 zombie Cpu0 : 86.7%us, 1.7%sy, 0.0%ni, 2.0%id, 9.7%wa, 0.0%hi, 0.0%si, 0.0%st Cpu1 : 96.7%us, 0.0%sy, 0.0%ni, 0.7%id, 2.7%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 2034284k total, 2018644k used, 15640k free, 2748k buffers Swap: 2097148k total, 24312k used, 2072836k free, 343256k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5861 fjp 20 0 1903m 1.4g 189m S 176 72.3 40:29.50 git