{"thread":{"id":"23310","subject":"Re: 'git gc --aggressive' effectively unusable","startedAt":"2010-04-02T22:05:35Z","lastAt":"2010-04-05T21:07:32Z","messageCount":13,"participants":["Frans Pop","Michael Witten","Miles Bader","Mike Galbraith","Michael Poole","Jeff King","Nicolas Pitre"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"138469","messageId":"201004030005.35737.elendil@planet.nl","threadId":"23310","inReplyTo":null,"subject":"'git gc --aggressive' effectively unusable","fromName":"Frans Pop","fromEmail":"elendil@planet.nl","sentAt":"2010-04-02T22:05:35Z","receivedAt":"2010-04-02T22:05:35Z","isPatch":false,"sender":{"key":"elendil@planet.nl","avatar":null},"body":"Note: this is on a different repo from the 'git reflog expire --all' I\nreported a bit earlier.\n\nI have a git-svn checkout of a subversion repo which I wanted to compress\nas much as possible. 'git gc --aggressive' starts to run fairly well, but\neats more and more memory and gets slower and slower. After it gets to\nabout 45% or 50% progress slows down noticeably and so far I haven't had\nthe patience to let it finish (40 minutes is already way too long).\n\nA regular 'git gc' run completes without any problems.\n\n$ du -sh .git/\n612M .git/\n\nSpecial about this repo is that it contains two huge objects [1], which\ncould maybe be a factor:\n size pack SHA\n- packages/po/sublevel4/da.po:\n 495661 4654 801cd6451ece536c0ab41f79e09fc52efdf3361f\n- packages/arch/powerpc/quik-installer/debian/po/da.po\n 149515 1403 83a787b20817dc4d72db052de4055e7a7c9221d7 \n\nBelow some output from top and of the progress of the command showing the\nproblem. Check the change in number of compressed objects against the\ntimestamps from top.\n\nCheers,\nFJP\n\n[1] Caused by a bug in a script a couple of years back.\n\n$ git gc --aggressive\n\nCounting objects: 843342, done.\nDelta compression using up to 2 threads.\nCompressing objects: 53% (449663/836424)\n\ntop - 22:55:02 up 18 min, 1 user, load average: 1.83, 1.68, 1.07\nTasks: 161 total, 1 running, 160 sleeping, 0 stopped, 0 zombie\nCpu0 : 91.4%us, 0.7%sy, 0.0%ni, 1.3%id, 6.6%wa, 0.0%hi, 0.0%si, 0.0%st\nCpu1 : 97.7%us, 0.3%sy, 0.0%ni, 1.3%id, 0.7%wa, 0.0%hi, 0.0%si, 0.0%st\nMem: 2034284k total, 2018288k used, 15996k free, 10188k buffers\nSwap: 2097148k total, 22612k used, 2074536k free, 449444k cached\n\n PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND\n 5861 fjp 20 0 1775m 1.3g 194m S 188 66.7 21:10.89 git\n\n\nCounting objects: 843342, done.\nDelta compression using up to 2 threads.\nCompressing objects: 58% (486001/836424)\n\ntop - 23:00:12 up 23 min, 1 user, load average: 1.96, 1.84, 1.30\nTasks: 158 total, 2 running, 156 sleeping, 0 stopped, 0 zombie\nCpu0 : 98.3%us, 0.7%sy, 0.0%ni, 0.7%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st\nCpu1 : 87.4%us, 1.7%sy, 0.0%ni, 0.0%id, 10.6%wa, 0.0%hi, 0.3%si, 0.0%st\nMem: 2034284k total, 2017516k used, 16768k free, 4696k buffers\nSwap: 2097148k total, 22572k used, 2074576k free, 336944k cached\n\n PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND\n 5861 fjp 20 0 1903m 1.4g 172m S 182 71.4 30:37.58 git\n\n\nCounting objects: 843342, done.\nDelta compression using up to 2 threads.\nCompressing objects: 61% (515958/836424)\n\ntop - 23:05:56 up 29 min, 1 user, load average: 1.68, 1.85, 1.48\nTasks: 159 total, 1 running, 158 sleeping, 0 stopped, 0 zombie\nCpu0 : 86.7%us, 1.7%sy, 0.0%ni, 2.0%id, 9.7%wa, 0.0%hi, 0.0%si, 0.0%st\nCpu1 : 96.7%us, 0.0%sy, 0.0%ni, 0.7%id, 2.7%wa, 0.0%hi, 0.0%si, 0.0%st\nMem: 2034284k total, 2018644k used, 15640k free, 2748k buffers\nSwap: 2097148k total, 24312k used, 2072836k free, 343256k cached\n\n PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND\n 5861 fjp 20 0 1903m 1.4g 189m S 176 72.3 40:29.50 git\n"},{"id":"138468","messageId":"201004030012.25092.elendil@planet.nl","threadId":"23310","inReplyTo":"201004030005.35737.elendil@planet.nl","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Frans Pop","fromEmail":"elendil@planet.nl","sentAt":"2010-04-02T22:12:24Z","receivedAt":"2010-04-02T22:12:24Z","isPatch":false,"sender":{"key":"elendil@planet.nl","avatar":null},"body":"The environment is the same as for the reflog problem, but I should have \nadded that info anyway. Here it is again.\n\nOn Saturday 03 April 2010, Frans Pop wrote:\n> I have a git-svn checkout of a subversion repo which I wanted to\n> compress as much as possible. 'git gc --aggressive' starts to run fairly\n> well, but eats more and more memory and gets slower and slower. After it\n> gets to about 45% or 50% progress slows down noticeably and so far I\n> haven't had the patience to let it finish (40 minutes is already way too\n> long).\n\nI'm seeing this with both git 1.6.6.1 and 1.7.0.3 on the same repo.\nEnvironment:\n- Debian amd64/Lenny; Core Duo x86_64 2.6.34-rc3 -> 1.6.6.1\n- Debian i386/Sid; chroot on the same machine -> 1.7.0.3\n"},{"id":"138513","messageId":"201004032316.22483.elendil@planet.nl","threadId":"23310","inReplyTo":"201004030005.35737.elendil@planet.nl","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Frans Pop","fromEmail":"elendil@planet.nl","sentAt":"2010-04-03T21:16:19Z","receivedAt":"2010-04-03T21:16:19Z","isPatch":false,"sender":{"key":"elendil@planet.nl","avatar":null},"body":"On Saturday 03 April 2010, Frans Pop wrote:\n> Special about this repo is that it contains two huge objects [1], which\n> could maybe be a factor:\n>      size    pack  SHA\n> - packages/po/sublevel4/da.po:\n>      495661  4654  801cd6451ece536c0ab41f79e09fc52efdf3361f\n> - packages/arch/powerpc/quik-installer/debian/po/da.po\n>      149515  1403  83a787b20817dc4d72db052de4055e7a7c9221d7  \n\nTo avoid confusion: these sizes are in kB.\n"},{"id":"138514","messageId":"p2gb4087cc51004031433xc57e52bbq733d1d3c5f37f238@mail.gmail.com","threadId":"23310","inReplyTo":"201004030005.35737.elendil@planet.nl","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Michael Witten","fromEmail":"mfwitten@gmail.com","sentAt":"2010-04-03T21:33:20Z","receivedAt":"2010-04-03T21:33:20Z","isPatch":false,"sender":{"key":"mfwitten@gmail.com","avatar":"https://avatars.githubusercontent.com/u/597101?v=4"},"body":"On Fri, Apr 2, 2010 at 16:05, Frans Pop wrote:\n> I haven't had the patience to let it finish\n\nThere's your problem.\n\n$ git help gc | sed -n /--aggressive$/,+3p\n --aggressive\n Usually git gc runs very quickly while\n providing good disk space utilization\n and performance. This option will\n cause git gc to more aggressively\n optimize the repository at the expense\n of taking much more time. The effects\n of this optimization are persistent, so\n this option only needs to be used\n occasionally; every few hundred\n changesets or so.\n\nLast time I used this option (on Linus's Linux repo), I let the\nalgorithm do its thing for a couple of hours. Maybe the efficiency\ncould be vastly improved, but it does finish if you let it.\n\nSIncerely,\nMichael Witten\n"},{"id":"138515","messageId":"g2kb4087cc51004031442jb2d825casc0b66454490288a@mail.gmail.com","threadId":"23310","inReplyTo":"p2gb4087cc51004031433xc57e52bbq733d1d3c5f37f238@mail.gmail.com","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Michael Witten","fromEmail":"mfwitten@gmail.com","sentAt":"2010-04-03T21:42:41Z","receivedAt":"2010-04-03T21:42:41Z","isPatch":false,"sender":{"key":"mfwitten@gmail.com","avatar":"https://avatars.githubusercontent.com/u/597101?v=4"},"body":"On Sat, Apr 3, 2010 at 15:33, Michael Witten wrote:\n> $ git help gc | sed -n /--aggressive$/,+3p\n\nAs an aside: I didn't realize I copied that in there; this would\nprobably be better:\n\n$ git help gc | sed -n /--aggressive$/,/^$/p\n"},{"id":"138518","messageId":"201004040123.06877.elendil@planet.nl","threadId":"23310","inReplyTo":"p2gb4087cc51004031433xc57e52bbq733d1d3c5f37f238@mail.gmail.com","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Frans Pop","fromEmail":"elendil@planet.nl","sentAt":"2010-04-03T23:23:04Z","receivedAt":"2010-04-03T23:23:04Z","isPatch":false,"sender":{"key":"elendil@planet.nl","avatar":null},"body":"On Saturday 03 April 2010, Michael Witten wrote:\n> On Fri, Apr 2, 2010 at 16:05, Frans Pop wrote:\n> > I haven't had the patience to let it finish\n>\n> There's your problem.\n\nYes, I had seen that. But there's a difference between taking much more \ntime and slowing down to such an extend that it never finishes.\n\nI've tried it today on my linux-2.6 repo as well and the same thing \nhappened. At first the progress is not fast but reasonable. When it gets \nto about 45% percent it starts slowing down a lot: from ~1500 objects per \nupdate of the counters to ~300 objects per update. And who knows what the \nprogress is going to be when it reaches 70% or 90%: 10 per update?\n\nWith a total of over 2 milion objects in the repository such a low speed is \nsimply not going to work, ever. So I maintain that it is effectively \nunusable.\n\nCheers,\nFJP\n"},{"id":"138519","messageId":"j2pb4087cc51004031642qa9444506s8d0d34e57e541953@mail.gmail.com","threadId":"23310","inReplyTo":"201004040123.06877.elendil@planet.nl","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Michael Witten","fromEmail":"mfwitten@gmail.com","sentAt":"2010-04-03T23:42:55Z","receivedAt":"2010-04-03T23:42:55Z","isPatch":false,"sender":{"key":"mfwitten@gmail.com","avatar":"https://avatars.githubusercontent.com/u/597101?v=4"},"body":"On Sat, Apr 3, 2010 at 17:23, Frans Pop wrote:\n> On Saturday 03 April 2010, Michael Witten wrote:\n>> On Fri, Apr 2, 2010 at 16:05, Frans Pop wrote:\n>> > I haven't had the patience to let it finish\n> ...\n> I've tried it today on my linux-2.6 repo as well and the same thing\n> happened. At first the progress is not fast but reasonable. When it gets\n> to about 45% percent it starts slowing down a lot: from ~1500 objects per\n> update of the counters to ~300 objects per update. And who knows what\n> the progress is going to be when it reaches 70% or 90%: 10 per update?\n>\n> With a total of over 2 milion objects in the repository such a low speed is\n> simply not going to work, ever. So I maintain that it is effectively\n> unusable.\n\nWell, all I can do is quote myself:\n\n Last time I used this option (on Linus's Linux repo),\n I let the algorithm do its thing for a couple of hours.\n Maybe the efficiency could be vastly improved, but\n it does finish if you let it.\n\nOn Fri, Apr 2, 2010 at 16:12, Frans Pop wrote:\n> I'm seeing this with both git 1.6.6.1 and 1.7.0.3 on the same repo.\n\nI think I must have run gc with 1.7.0.2.199.g90a2bf9; perhaps you\ncould use something like oprofile to figure out where gc is spending\nmost of its time.\n"},{"id":"138520","messageId":"87oci0m5v4.fsf@catnip.gol.com","threadId":"23310","inReplyTo":"201004040123.06877.elendil@planet.nl","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Miles Bader","fromEmail":"miles@gnu.org","sentAt":"2010-04-04T00:14:07Z","receivedAt":"2010-04-04T00:14:07Z","isPatch":false,"sender":{"key":"miles@gnu.org","avatar":"https://gravatar.com/avatar/01069b69593af7bff28e2f97afeb3644ae6fe2f5f56cb3a8cf34c5fb8c36efe5?d=mp&s=160"},"body":"Frans Pop writes:\n>> > I haven't had the patience to let it finish\n>>\n>> There's your problem.\n>\n> Yes, I had seen that. But there's a difference between taking much more \n> time and slowing down to such an extend that it never finishes.\n>\n> I've tried it today on my linux-2.6 repo as well and the same thing \n> happened. At first the progress is not fast but reasonable. When it gets \n> to about 45% percent it starts slowing down a lot: from ~1500 objects per \n> update of the counters to ~300 objects per update. And who knows what the \n> progress is going to be when it reaches 70% or 90%: 10 per update?\n\nAre you sure it doesn't subsequently speed up again?\n\n-Miles\n\n-- \nIdiot, n. A member of a large and powerful tribe whose influence in human\naffairs has always been dominant and controlling.\n"},{"id":"138524","messageId":"1270355267.6307.43.camel@marge.simson.net","threadId":"23310","inReplyTo":"201004040123.06877.elendil@planet.nl","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Mike Galbraith","fromEmail":"efault@gmx.de","sentAt":"2010-04-04T04:27:47Z","receivedAt":"2010-04-04T04:27:47Z","isPatch":false,"sender":{"key":"efault@gmx.de","avatar":null},"body":"On Sun, 2010-04-04 at 01:23 +0200, Frans Pop wrote:\n> On Saturday 03 April 2010, Michael Witten wrote:\n> > On Fri, Apr 2, 2010 at 16:05, Frans Pop wrote:\n> > > I haven't had the patience to let it finish\n> >\n> > There's your problem.\n> \n> Yes, I had seen that. But there's a difference between taking much more \n> time and slowing down to such an extend that it never finishes.\n> \n> I've tried it today on my linux-2.6 repo as well and the same thing \n> happened. At first the progress is not fast but reasonable. When it gets \n> to about 45% percent it starts slowing down a lot: from ~1500 objects per \n> update of the counters to ~300 objects per update. And who knows what the \n> progress is going to be when it reaches 70% or 90%: 10 per update?\n> \n> With a total of over 2 milion objects in the repository such a low speed is \n> simply not going to work, ever. So I maintain that it is effectively \n> unusable.\n\nAs a data point, when I do gc, I routinely use --aggressive. It takes a\nwhile here, but not forever. (I'm a tad short of 2 million objects)\n\nRepo is mainline + next + tip + stable >= 2.6.22 + local branches.\n\ngit@marge:..git/linux-2.6> time git gc --aggressive\nCounting objects: 1909894, done.\nDelta compression using up to 4 threads.\nCompressing objects: 100% (1889774/1889774), done.\nWriting objects: 100% (1909894/1909894), done.\nTotal 1909894 (delta 1674098), reused 0 (delta 0)\n\nreal 22m24.943s\nuser 55m33.756s\nsys 0m8.149s\n\ngit is 1.7.0.3\n\n\t-Mike\n"},{"id":"138540","messageId":"87zl1js248.fsf@troilus.org","threadId":"23310","inReplyTo":"87oci0m5v4.fsf@catnip.gol.com","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Michael Poole","fromEmail":"mdpoole@troilus.org","sentAt":"2010-04-04T14:50:47Z","receivedAt":"2010-04-04T14:50:47Z","isPatch":false,"sender":{"key":"mdpoole@troilus.org","avatar":null},"body":"Miles Bader writes:\n\n> Frans Pop writes:\n>>> > I haven't had the patience to let it finish\n>>>\n>>> There's your problem.\n>>\n>> Yes, I had seen that. But there's a difference between taking much more \n>> time and slowing down to such an extend that it never finishes.\n>>\n>> I've tried it today on my linux-2.6 repo as well and the same thing \n>> happened. At first the progress is not fast but reasonable. When it gets \n>> to about 45% percent it starts slowing down a lot: from ~1500 objects per \n>> update of the counters to ~300 objects per update. And who knows what the \n>> progress is going to be when it reaches 70% or 90%: 10 per update?\n>\n> Are you sure it doesn't subsequently speed up again?\n\nI have seen asymptotic slowdown as \"git gc --aggressive\" progresses on\ncertain repositories. It is particularly bad with\ngit://git.infradead.org/gcc.git (on an x86-64 system with 4 GB RAM).\ngit seemed to be thrashing swap badly as time went on. I don't know\nthat git gc --aggressive would *never* finish on my gcc-git repository.\nI just know that it got to about 80% done in less than an hour, to 90%\nafter twelve hours, and about 94% after another twelve hours. (The same\noperation on linux-2.6.git takes about 40 minutes with all the default\nsettings.)\n\nI may have been dreaming, but I thought with some 1.6.x version of git,\nreducing core.packedGitLimit and pack.windowLimit (now windowMemory?)\nmostly made the thrashing go away. When I try again with v1.7.0.2,\nthough, it doesn't seem to help very much -- there is still a lot of\nswapping, and the git process got to about 7 GB virtual size before I\nkilled it after about 10 hours of operation.\n\nMichael Poole\n"},{"id":"138554","messageId":"20100404203850.GA8798@coredump.intra.peff.net","threadId":"23310","inReplyTo":"87zl1js248.fsf@troilus.org","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-04-04T20:38:51Z","receivedAt":"2010-04-04T20:38:51Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Apr 04, 2010 at 10:50:47AM -0400, Michael Poole wrote:\n\n> > Are you sure it doesn't subsequently speed up again?\n> \n> I have seen asymptotic slowdown as \"git gc --aggressive\" progresses on\n> certain repositories. It is particularly bad with\n> git://git.infradead.org/gcc.git (on an x86-64 system with 4 GB RAM).\n> git seemed to be thrashing swap badly as time went on. I don't know\n> that git gc --aggressive would *never* finish on my gcc-git repository.\n> I just know that it got to about 80% done in less than an hour, to 90%\n> after twelve hours, and about 94% after another twelve hours. (The same\n> operation on linux-2.6.git takes about 40 minutes with all the default\n> settings.)\n> \n> I may have been dreaming, but I thought with some 1.6.x version of git,\n> reducing core.packedGitLimit and pack.windowLimit (now windowMemory?)\n> mostly made the thrashing go away. When I try again with v1.7.0.2,\n> though, it doesn't seem to help very much -- there is still a lot of\n> swapping, and the git process got to about 7 GB virtual size before I\n> killed it after about 10 hours of operation.\n\nI packed Frans' sample kernel repo with \"git gc --aggressive\" last\nnight. It did finish after about 9 hours. I didn't take memory usage\nmeasurements, but here's what time said:\n\n real 535m38.898s\n user 216m46.437s\n sys 0m24.186s\n\nThat's 3.6 hours of CPU time over almost 9 hours (on a dual-core\nmachine). The non-agressive pack was about 680M, and the result was\n480M. The machine has 2G of RAM, and not much else running. So I would\nreally not expect there to be much disk I/O required, but clearly we\nwere waiting quite a bit.\n\nI'll try tweaking a few of the pack memory limits and try again.\n\n-Peff\n"},{"id":"138562","messageId":"20100404214944.GA15104@coredump.intra.peff.net","threadId":"23310","inReplyTo":"20100404203850.GA8798@coredump.intra.peff.net","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2010-04-04T21:49:44Z","receivedAt":"2010-04-04T21:49:44Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, Apr 04, 2010 at 04:38:50PM -0400, Jeff King wrote:\n\n> I packed Frans' sample kernel repo with \"git gc --aggressive\" last\n> night. It did finish after about 9 hours. I didn't take memory usage\n> measurements, but here's what time said:\n> \n> real 535m38.898s\n> user 216m46.437s\n> sys 0m24.186s\n> \n> That's 3.6 hours of CPU time over almost 9 hours (on a dual-core\n> machine). The non-agressive pack was about 680M, and the result was\n> 480M. The machine has 2G of RAM, and not much else running. So I would\n> really not expect there to be much disk I/O required, but clearly we\n> were waiting quite a bit.\n> \n> I'll try tweaking a few of the pack memory limits and try again.\n\nHmm, this may be relevant:\n\n http://thread.gmane.org/gmane.comp.version-control.git/67791/focus=94797\n\nIn my experiments, memory usage is increasing but valgrind doesn't\nleaks. So perhaps it is fragmentation in the memory allocator.\n\n-Peff\n"},{"id":"138675","messageId":"alpine.LFD.2.00.1004051615070.7232@xanadu.home","threadId":"23310","inReplyTo":"20100404214944.GA15104@coredump.intra.peff.net","subject":"Re: 'git gc --aggressive' effectively unusable","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-04-05T21:07:32Z","receivedAt":"2010-04-05T21:07:32Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sun, 4 Apr 2010, Jeff King wrote:\n\n> On Sun, Apr 04, 2010 at 04:38:50PM -0400, Jeff King wrote:\n> \n> > I packed Frans' sample kernel repo with \"git gc --aggressive\" last\n> > night. It did finish after about 9 hours. I didn't take memory usage\n> > measurements, but here's what time said:\n> > \n> > real 535m38.898s\n> > user 216m46.437s\n> > sys 0m24.186s\n> > \n> > That's 3.6 hours of CPU time over almost 9 hours (on a dual-core\n> > machine). The non-agressive pack was about 680M, and the result was\n> > 480M. The machine has 2G of RAM, and not much else running. So I would\n> > really not expect there to be much disk I/O required, but clearly we\n> > were waiting quite a bit.\n> > \n> > I'll try tweaking a few of the pack memory limits and try again.\n> \n> Hmm, this may be relevant:\n> \n> http://thread.gmane.org/gmane.comp.version-control.git/67791/focus=94797\n> \n> In my experiments, memory usage is increasing but valgrind doesn't\n> leaks. So perhaps it is fragmentation in the memory allocator.\n\nTo verify this, simply try with pack.threads = 1. That should help the \nmemory allocator not to fragment memory allocation across threads \nrandomly.\n\nAlso, going multithreaded _may_ be faster only if you can afford the \nincreased memory usage. Especially with gc --aggressive, each thread is \nadding its own share of memory usage in the delta window.\n\nFirst thing to try for the biggest possible improvement is \npack.threads=1. On a quad core machine this means repacking 4 times \nslower, but this is certainly much faster than 100 times slower when the \nsystem starts swapping. That might even make the resulting pack a tad \ntighter due to delta windows not being fragmented across different \nthreads.\n\nIf that is not enough, then try:\n\n\tpack.deltaCacheSize = 1\n\tcore.packedGitWindowSize = 16m\n\tcore.packedGitLimit = 128m\n\nThis should reduce Git's memory usage while making it slower without \naffecting the packing outcome. Again \"slower\" could mean \"much faster\" \nif by reducing memory usage then swapping is completely avoided.\n\nIf that still doesn't help much, then the next tweaks will affect the \npacking result:\n\n\tpack.windowMemory = 256m\n\nHere 256m is arbitrary and must be guessed from the size of the objects \nbeing packed. The idea is to let smallish objects completely fill the \nsearch window (it has 250 entries by default with --aggressive) while \nnot letting that many huge objects completely eat up all memory. If \nthere is still swapping going on then you can try 64m instead. That \nmeans that if you have a large set of 1MB objects then the delta search \nwindow will be scaled down to less than 64 entries in that case. This \nis why packing might be less optimal as there are fewer delta \ncombinations being considered.\n\nIf this still doesn't prevent swapping then you should really consider \ninstalling more RAM. There are fundamental object accounting structures \nthat can hardly be shrunk such as struct object_entry in \nbuiltin/pack-objects.c, and one instance of such structure is needed for \neach object. On a 64-bit machine this structure occupies 120 bytes, \nmeaning 2M objects requires 240MB of RAM just for that. The data set \nalso has to fit in the file cache to avoid IO trashing. So if your \nrepository is larger than the available RAM then some trashing is almost \nunavoidable. Sometimes a badly packed repository may require 2GB of \ndisk space in the .git directory alone while the fully packed version is \nonly a few hundred megabytes. Such repositories may need to be repacked \non a big machine first, before machines with less RAM are able to handle \nit afterwards.\n\nHope this helps.\n\n\nNicolas\n"}]}