{"thread":{"id":"28560","subject":"pack-object poor performance (with large number of objects?)","startedAt":"2011-10-03T14:43:02Z","lastAt":"2011-10-05T08:48:25Z","messageCount":11,"participants":["Piotr Krukowiecki","Shawn Pearce","Junio C Hamano","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"176746","messageId":"CAA01CspZijOO_xbR=OcaRaesTeSy=6RM4DR01-07qimVzxvJZA@mail.gmail.com","threadId":"28560","inReplyTo":null,"subject":"pack-object poor performance (with large number of objects?)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2011-10-03T14:43:02Z","receivedAt":"2011-10-03T14:43:02Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"Hi,\n\nI'm having poor git gc (pack-object) performance. Please read below\nfor details. What can I do to improve the performance/debug the reason\nfor the slowness? Should I leave the process running over night, or\nshould I stop it (for debugging)?\nCCing people who posted some patches/benchmarks for pack-objects recently.\n\ngit gc was first run automatically by git svn clone. It found 1544673\nobjects and worked for 50 minutes until I've killed it.\n\nThen I've run it by hand with --aggresive (because I've found on\nInternet it helped in some cases). It found 1742200 objects this time.\nAt this moment it's been working for about 90 minutes.\n\nThe large number of unpacked objects is probably caused by me - I've\ndisabled auto gc when I was cloning from svn (I though it might speed\nup things if it didn't repack several times during clone, only once\nafterwards).\n\nMy git version is 1.7.7.rc3.4.g8d714\nThe file system is ext4.\n\nFirst run process tree:\npkruk 27873 0.0 0.0 15704 816 pts/2 S+ 11:53 0:00\n | | \\_ git gc --auto\npkruk 27885 0.0 0.0 15704 776 pts/2 S+ 11:53 0:00\n | | \\_ git repack -d -l\npkruk 27886 0.0 0.0 4220 608 pts/2 S+ 11:53 0:00\n | | \\_ /bin/sh\n/usr/local/stow/git-master/libexec/git-core/git-repack -d -l\npkruk 27897 3.6 9.3 1136072 381148 pts/2 D+ 11:53 5:51\n | | \\_ git\npack-objects --keep-true-parents --honor-pack-keep --non-empty --all\n--reflog --unpacked --incremental --local --delta-base-offset\n/home/pkruk/dv/devel1_git_repos/.git/objects/pack/.tmp-27886-pack\n\nSecond run process tree:\npkruk 6171 0.0 0.0 15704 1428 pts/2 S+ 14:34 0:00\n | | \\_ git gc --aggressive\npkruk 6174 0.0 0.0 15704 1356 pts/2 S+ 14:34 0:00\n | | \\_ git repack -d -l -f\n--depth=250 --window=250 -A\npkruk 6175 0.0 0.0 4220 648 pts/2 S+ 14:34 0:00\n | | \\_ /bin/sh\n/usr/local/stow/git-master/libexec/git-core/git-repack -d -l -f\n--depth=250 --window=250 -A\npkruk 6189 4.9 10.5 1143640 427396 pts/2 D+ 14:34 4:50\n | | \\_ git pack-objects\n--keep-true-parents --honor-pack-keep --non-empty --all --reflog\n--unpack-unreachable --local --no-reuse-delta --depth=250 --window=250\n--delta-base-offset\n/home/pkruk/dv/devel1_git_repos/.git/objects/pack/.tmp-6175-pack\n\n\n\n-- \nPiotr Krukowiecki\n"},{"id":"176749","messageId":"CAJo=hJtw+sYrP09zrDbZJNGHDYOeguQLkOe88FBYQDZrnaqsAw@mail.gmail.com","threadId":"28560","inReplyTo":"CAA01CspZijOO_xbR=OcaRaesTeSy=6RM4DR01-07qimVzxvJZA@mail.gmail.com","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Shawn Pearce","fromEmail":"spearce@spearce.org","sentAt":"2011-10-03T16:05:06Z","receivedAt":"2011-10-03T16:05:06Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"On Mon, Oct 3, 2011 at 07:43, Piotr Krukowiecki\n wrote:\n> I'm having poor git gc (pack-object) performance. Please read below\n> for details. What can I do to improve the performance/debug the reason\n> for the slowness? Should I leave the process running over night, or\n> should I stop it (for debugging)?\n> CCing people who posted some patches/benchmarks for pack-objects recently.\n>\n> git gc was first run automatically by git svn clone. It found 1544673\n> objects and worked for 50 minutes until I've killed it.\n>\n> Then I've run it by hand with --aggresive (because I've found on\n> Internet it helped in some cases). It found 1742200 objects this time.\n> At this moment it's been working for about 90 minutes.\n\nPacking time depends on a number of factors. One of them is the number\nof unpacked objects to process. With 1.7 million objects, yes, its\ngoing to take some time. Another factor is how much RAM you have on\nyour system. Packing requires a lot of memory, especially with the\n--aggressive flag, as the packer tries up to 250 different\ncombinations of two objects searching for a good delta compression\nformat, and all 250 of those are typically in-memory at once. If you\nhave insufficient physical RAM, the system will swap, unless you\ndecrease the window size.\n\n> The large number of unpacked objects is probably caused by me - I've\n> disabled auto gc when I was cloning from svn (I though it might speed\n> up things if it didn't repack several times during clone, only once\n> afterwards).\n\nYes, this the reason `git svn` runs GC during its import. If you defer\nall of the repacking work until the end, with everything loose, it can\ntake a very, very long time to repack. If you repack as you go, the\nincremental repacks are less expensive than a full repack, and the\nentire process will go faster overall.\n\n-- \nShawn.\n"},{"id":"176753","messageId":"CAA01CsppPf_6Zp5UPYBsxa1JEwLGF-FqacRa7kBJ45Ges2ujrw@mail.gmail.com","threadId":"28560","inReplyTo":"CAJo=hJtw+sYrP09zrDbZJNGHDYOeguQLkOe88FBYQDZrnaqsAw@mail.gmail.com","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2011-10-03T17:17:19Z","receivedAt":"2011-10-03T17:17:19Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Mon, Oct 3, 2011 at 6:05 PM, Shawn Pearce wrote:\n> On Mon, Oct 3, 2011 at 07:43, Piotr Krukowiecki\n> wrote:\n>> I'm having poor git gc (pack-object) performance. Please read below\n>> for details. What can I do to improve the performance/debug the reason\n>> for the slowness? Should I leave the process running over night, or\n>> should I stop it (for debugging)?\n>> CCing people who posted some patches/benchmarks for pack-objects recently.\n>>\n>> git gc was first run automatically by git svn clone. It found 1544673\n>> objects and worked for 50 minutes until I've killed it.\n>>\n>> Then I've run it by hand with --aggresive (because I've found on\n>> Internet it helped in some cases). It found 1742200 objects this time.\n>> At this moment it's been working for about 90 minutes.\n>\n> Packing time depends on a number of factors. One of them is the number\n> of unpacked objects to process. With 1.7 million objects, yes, its\n> going to take some time.\n\nAny statistics how long it should take?\n\n\n> Another factor is how much RAM you have on\n> your system. Packing requires a lot of memory, especially with the\n> --aggressive flag, as the packer tries up to 250 different\n> combinations of two objects searching for a good delta compression\n> format, and all 250 of those are typically in-memory at once. If you\n> have insufficient physical RAM, the system will swap, unless you\n> decrease the window size.\n\nI have 4GB of RAM and not all was used so it certainly shouldn't be\nswapping. The process was in 'D' state so I suppose the hard disk\nmight be the limiting factor.\n\nI think I also disabled threading (I'll check tomorrow) - I suppose it\nhas impact on packing time too.\n\nI'll re-run packing tomorrow with threading and check the memory\nusage, is there anything else I can do?\n\n>\n>> The large number of unpacked objects is probably caused by me - I've\n>> disabled auto gc when I was cloning from svn (I though it might speed\n>> up things if it didn't repack several times during clone, only once\n>> afterwards).\n>\n> Yes, this the reason `git svn` runs GC during its import. If you defer\n> all of the repacking work until the end, with everything loose, it can\n> take a very, very long time to repack. If you repack as you go, the\n> incremental repacks are less expensive than a full repack, and the\n> entire process will go faster overall.\n\nI've learned the lesson :)\n\n-- \nPiotr Krukowiecki\n"},{"id":"176781","messageId":"7v62k5g988.fsf@alter.siamese.dyndns.org","threadId":"28560","inReplyTo":"CAA01CsppPf_6Zp5UPYBsxa1JEwLGF-FqacRa7kBJ45Ges2ujrw@mail.gmail.com","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-10-03T19:34:31Z","receivedAt":"2011-10-03T19:34:31Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Piotr Krukowiecki writes:\n\n>> Packing time depends on a number of factors. One of them is the number\n>> of unpacked objects to process. With 1.7 million objects, yes, its\n>> going to take some time.\n>\n> Any statistics how long it should take?\n\nPacking time depends on the repository, your machine and how you pack, so\nsuch statistics would be useful only in comparable contexts.\n\n linux-3.0/master$ time git repack -a -d \n Counting objects: 2138578, done.\n Delta compression using up to 4 threads.\n Compressing objects: 100% (327257/327257), done.\n Writing objects: 100% (2138578/2138578), done.\n Total 2138578 (delta 1791983), reused 2138009 (delta 1791434)\n\n real 1m40.528s\n user 1m22.805s\n sys 0m3.788s\n linux-3.0/master$ git count-objects -v\n count: 0\n size: 0\n in-pack: 2138578\n packs: 1\n size-pack: 487957\n prune-packable: 0\n garbage: 0\n\nThis is on my box [*1*] that is idle (other than running the repack). The\nabove is starting from an already reasonably well packed state and reuses\ndeltas; with \"-f\" to repack everything from scratch it would take\nsignificantly longer:\n\n linux-3.0/master$ time git repack -a -d -f\n Counting objects: 2138578, done.\n Delta compression using up to 4 threads.\n Compressing objects: 100% (2118691/2118691), done.\n Writing objects: 100% (2138578/2138578), done.\n Total 2138578 (delta 1749156), reused 344219 (delta 0)\n\n real 3m26.750s\n user 8m41.857s\n sys 0m6.716s\n\nLarger \"window\" tends to make the process take longer (I think it grows\nsquared) but may reduce both the resulting packsize and runtime access\noverhead. Larger \"depth\" does not affect time to pack and helps reducing\nthe resulting packsize, with increased runtime access overhead (i.e. not\nreally recommended).\n\n[Footnote]\n\n*1* http://gitster.livejournal.com/34818.html\n\nIntel(R) Core(TM)2 Quad CPU Q9450 @ 2.66GHz with 8GB memory.\n"},{"id":"176820","messageId":"CAA01CsoSTsBLNcbv5o6Jx6YrjG4g8T=yodX811ymBLXg7sjDJQ@mail.gmail.com","threadId":"28560","inReplyTo":"7v62k5g988.fsf@alter.siamese.dyndns.org","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2011-10-04T07:59:08Z","receivedAt":"2011-10-04T07:59:08Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Mon, Oct 3, 2011 at 9:34 PM, Junio C Hamano wrote:\n> This is on my box [*1*] that is idle (other than running the repack). The\n> above is starting from an already reasonably well packed state and reuses\n> deltas; with \"-f\" to repack everything from scratch it would take\n> significantly longer:\n>\n>    linux-3.0/master$ time git repack -a -d -f\n>    Counting objects: 2138578, done.\n>    Delta compression using up to 4 threads.\n>    Compressing objects: 100% (2118691/2118691), done.\n>    Writing objects: 100% (2138578/2138578), done.\n>    Total 2138578 (delta 1749156), reused 344219 (delta 0)\n>\n>    real    3m26.750s\n>    user    8m41.857s\n>    sys     0m6.716s\n\nI've run the command and it took about 20 minutes in \"Counting\nobjects\" to count up to 500000 on idle machine and there's still 700MB\nRAM free.\n\nI wonder, when you do the repacking from a packed state, does it\nphysically create files on file system? In my case I have lots of\nfiles in objects dir:\n\n$ ls objects/ | wc -l\n258\n$ ls objects/00 | wc -l\n6173\n\nWhen I tried 'find objects | wc -l' previously (when repack was not\nrunning) it got \"stuck\" too and I got impatient and killed it ;)\n\nSo it looks it's not a problem with git but rather with my disk/file\nsystem/linux...\n\n\n-- \nPiotr Krukowiecki\n"},{"id":"176837","messageId":"20111004110702.GA18599@sigill.intra.peff.net","threadId":"28560","inReplyTo":"CAA01CsoSTsBLNcbv5o6Jx6YrjG4g8T=yodX811ymBLXg7sjDJQ@mail.gmail.com","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-10-04T11:07:02Z","receivedAt":"2011-10-04T11:07:02Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 04, 2011 at 09:59:08AM +0200, Piotr Krukowiecki wrote:\n\n> I've run the command and it took about 20 minutes in \"Counting\n> objects\" to count up to 500000 on idle machine and there's still 700MB\n> RAM free.\n> [...]\n> So it looks it's not a problem with git but rather with my disk/file\n> system/linux...\n\nYou mentioned that git was in the 'D' state earlier. And it sounds like\nyou have 1.7 million objects, _completely_ unpacked.\n\nSo my guess is that it is simply taking an enormous amount of disk\nspace, and git is mostly waiting on the disk to read in files. What does\n\"du -sh .git/objects\" say?\n\n-Peff\n"},{"id":"176843","messageId":"CAA01CsodyUQJOnj5vV0LdVEWpkvwSW2TAONzyY9J82o9VwC6Ag@mail.gmail.com","threadId":"28560","inReplyTo":"20111004110702.GA18599@sigill.intra.peff.net","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2011-10-04T12:22:55Z","receivedAt":"2011-10-04T12:22:55Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Tue, Oct 4, 2011 at 1:07 PM, Jeff King wrote:\n> On Tue, Oct 04, 2011 at 09:59:08AM +0200, Piotr Krukowiecki wrote:\n>\n>> I've run the command and it took about 20 minutes in \"Counting\n>> objects\" to count up to 500000 on idle machine and there's still 700MB\n>> RAM free.\n>> [...]\n>> So it looks it's not a problem with git but rather with my disk/file\n>> system/linux...\n>\n> You mentioned that git was in the 'D' state earlier. And it sounds like\n> you have 1.7 million objects, _completely_ unpacked.\n\nThat's right - since I had auto-gc disabled at first it had not chance\nto pack anything.\n\n\n> So my guess is that it is simply taking an enormous amount of disk\n> space, and git is mostly waiting on the disk to read in files. What does\n> \"du -sh .git/objects\" say?\n\nIt isn't that big - it's 11G.\n.git/objects/pack/ is 666MB currently.\n\n\n-- \nPiotr Krukowiecki\n"},{"id":"176845","messageId":"20111004124502.GB30162@sigill.intra.peff.net","threadId":"28560","inReplyTo":"CAA01CsodyUQJOnj5vV0LdVEWpkvwSW2TAONzyY9J82o9VwC6Ag@mail.gmail.com","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-10-04T12:45:02Z","receivedAt":"2011-10-04T12:45:02Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 04, 2011 at 02:22:55PM +0200, Piotr Krukowiecki wrote:\n\n> > So my guess is that it is simply taking an enormous amount of disk\n> > space, and git is mostly waiting on the disk to read in files. What does\n> > \"du -sh .git/objects\" say?\n> \n> It isn't that big - it's 11G.\n> .git/objects/pack/ is 666MB currently.\n\nBut you have 4G of RAM, no? So depending on the access patterns, you are\nthrashing your disk cache and always pulling each object straight from\ndisk.\n\n-Peff\n"},{"id":"176846","messageId":"CAA01Csp2rouKk4jvCH0Wu+0gc3+cvyH__d-yw8EHEkeZhRpX1Q@mail.gmail.com","threadId":"28560","inReplyTo":"20111004124502.GB30162@sigill.intra.peff.net","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2011-10-04T13:21:24Z","receivedAt":"2011-10-04T13:21:24Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Tue, Oct 4, 2011 at 2:45 PM, Jeff King wrote:\n> On Tue, Oct 04, 2011 at 02:22:55PM +0200, Piotr Krukowiecki wrote:\n>\n>> > So my guess is that it is simply taking an enormous amount of disk\n>> > space, and git is mostly waiting on the disk to read in files. What does\n>> > \"du -sh .git/objects\" say?\n>>\n>> It isn't that big - it's 11G.\n>> .git/objects/pack/ is 666MB currently.\n>\n> But you have 4G of RAM, no? So depending on the access patterns, you are\n> thrashing your disk cache and always pulling each object straight from\n> disk.\n\nI have 4GB ram + 4GB swap. Is it possible the RAM is the problem if I\nalways have free RAM left and my swap is almost not used?\nFor example at the moment repack finished counting objects (\"Counting\nobjects: 1742200, done.\"):\n\n$ free -m\n total used free shared buffers cached\nMem: 3960 3814 146 0 441 215\n-/+ buffers/cache: 3157 803\nSwap: 6143 694 5449\n\n$ ps auxwwww | grep git\npkruk 13541 0.0 0.0 15704 716 pts/2 S+ 13:19 0:00 git\nrepack -a -d -f\npkruk 13542 0.0 0.0 4220 540 pts/2 S+ 13:19 0:00\n/bin/sh /usr/local/stow/git-master/libexec/git-core/git-repack -a -d\n-f\npkruk 13556 3.9 9.8 1143628 401232 pts/2 DN+ 13:19 4:25 git\npack-objects --keep-true-parents --honor-pack-keep --non-empty --all\n--reflog --no-reuse-delta --delta-base-offset\n/home/pkruk/dv/devel1_git_repos/.git/objects/pack/.tmp-13542-pack\n\n\nI have updated to 1.7.7 btw.\n\n-- \nPiotr Krukowiecki\n"},{"id":"176861","messageId":"20111004180829.GB31671@sigill.intra.peff.net","threadId":"28560","inReplyTo":"CAA01Csp2rouKk4jvCH0Wu+0gc3+cvyH__d-yw8EHEkeZhRpX1Q@mail.gmail.com","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-10-04T18:08:29Z","receivedAt":"2011-10-04T18:08:29Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Tue, Oct 04, 2011 at 03:21:24PM +0200, Piotr Krukowiecki wrote:\n\n> I have 4GB ram + 4GB swap. Is it possible the RAM is the problem if I\n> always have free RAM left and my swap is almost not used?\n> For example at the moment repack finished counting objects (\"Counting\n> objects: 1742200, done.\"):\n> \n> $ free -m\n> total used free shared buffers cached\n> Mem: 3960 3814 146 0 441 215\n> -/+ buffers/cache: 3157 803\n> Swap: 6143 694 5449\n\nI am not the best person to comment on Linux's disk caching strategies,\nbut in general, it should prefer dropping disk cache over pushing\nprogram memory into swap. So no, you're not swapping, but you are\nworking with only 800M or so to do your disk caching.\n\nSo depending how big pack-object's working set of objects is, we might\nbe overflowing that, and constantly evicting and re-reading objects. I\ndon't recall offhand what kind of locality there is to pack-object's\naccesses.\n\nOne thing you could try to reduce the working set is to incrementally\npack some smaller chunks, and then combine them all at the end. That\nends up being more work overall, but at any given time, your working set\nof objects will be smaller.\n\nYou'd have to do something like this (this is very untested):\n\n # find out how many revisions we have. Let's pretend it's about\n # 25,000.\n git rev-list HEAD | wc -l\n\n # now split them into chunks of whatever size you feel like trying.\n # 1000, maybe, or a few thousand. Bearing in mind that this is a gross\n # approximation, since the history is not linear.\n #\n # Start with HEAD~24K (25K total, minus 1K we want to pack)\n echo HEAD~24000 | git pack-objects --revs .git/objects/pack/pack\n # And then prune the loose objects that we just packed.\n git prune-packed\n # And repeat for the next chunk\n echo HEAD~24000..HEAD~23000 | git pack-objects --revs .git/objects/pack/pack\n git prune-packed\n # And so forth...\n\nAnd then at the end, probably do a \"git repack -ad\" to put it all in\none big pack. Which should hopefully be less disk-intensive, because now\nyou'll have a much smaller disk footprint, since most of your objects\nare at least delta'd against the others in their own pack.\n\nI have no idea if this will actually go faster for you. But it might be\nworth trying, instead of just redoing the svn import with auto-gc turned\non.\n\n-Peff\n"},{"id":"176914","messageId":"CAA01CsoZVhbZHQgCsbk=XV833oQCbw-SVv9FuPQ05DFu49c0sQ@mail.gmail.com","threadId":"28560","inReplyTo":"20111004180829.GB31671@sigill.intra.peff.net","subject":"Re: pack-object poor performance (with large number of objects?)","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2011-10-05T08:48:25Z","receivedAt":"2011-10-05T08:48:25Z","isPatch":false,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Tue, Oct 4, 2011 at 8:08 PM, Jeff King wrote:\n> On Tue, Oct 04, 2011 at 03:21:24PM +0200, Piotr Krukowiecki wrote:\n>\n>> I have 4GB ram + 4GB swap. Is it possible the RAM is the problem if I\n>> always have free RAM left and my swap is almost not used?\n>> For example at the moment repack finished counting objects (\"Counting\n>> objects: 1742200, done.\"):\n>>\n>> $ free -m\n>>              total       used       free     shared    buffers     cached\n>> Mem:          3960       3814        146          0        441        215\n>> -/+ buffers/cache:       3157        803\n>> Swap:         6143        694       5449\n>\n[...]\n> I have no idea if this will actually go faster for you. But it might be\n> worth trying, instead of just redoing the svn import with auto-gc turned\n> on.\n\nI've left it to run over night and it finished (took almost 12 hours),\nso hopefully I'm not going to run into this problem anymore.\n\n$ time git repack -a -d -f\nCounting objects: 1742200, done.\nDelta compression using up to 4 threads.\nCompressing objects: 100% (1291909/1291909), done.\nWriting objects: 100% (1742200/1742200), done.\nTotal 1742200 (delta 1094325), reused 39192 (delta 0)\nRemoving duplicate objects: 100% (256/256), done.\n\nreal\t704m3.477s\nuser\t65m35.960s\nsys\t9m50.880s\n\n$ du -sh .git/objects/pack\n3.9G\t.git/objects/pack\n\n$ git count-objects -v\ncount: 0\nsize: 0\nin-pack: 1742200\npacks: 1\nsize-pack: 4078245\nprune-packable: 0\ngarbage: 0\n\n\n-- \nPiotr Krukowiecki\n"}]}