{"thread":{"id":"22534","subject":"git gc / git repack not removing unused objects?","startedAt":"2010-02-05T19:45:41Z","lastAt":"2010-02-08T17:12:03Z","messageCount":10,"participants":["Jon Nelson","Nicolas Pitre"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"133705","messageId":"cccedfc61002051145q1ff673e7s3db3bd7290be25e1@mail.gmail.com","threadId":"22534","inReplyTo":null,"subject":"git gc / git repack not removing unused objects?","fromName":"Jon Nelson","fromEmail":"jnelson@jamponi.net","sentAt":"2010-02-05T19:45:41Z","receivedAt":"2010-02-05T19:45:41Z","isPatch":false,"sender":{"key":"jnelson@jamponi.net","avatar":null},"body":"[Using git 1.6.4.2]\n\nIn one repo I have (136G objects directory, fully packed) I'm having\nsome trouble.\nI've run git-gc --prune=now, git repack -Adf, and so on a half-dozen\ntimes and each time I do so it gets bigger, not smaller.\nSetting that aside for the moment, however, I've run into a stranger problem.\n\nSo I use \"git verify-pack -v > gvp.out\" and \"sort -k3nr < gvp.out |\nhead -n 20\" to find the top 20 largest blobs.\nSo I have a blob, b32c3d8e8e24d8d3035cf52f606c2873315fe2b8, and now I\nwant to know what tree (or trees) it is in, so I try this:\n\n\nfor i in $( git branch -a | sed -e 's/\\*//g' | grep -v branch ); do if\ngit ls-tree -l -r -t $i | grep\nb32c3d8e8e24d8d3035cf52f606c2873315fe2b8 > /dev/null; then echo $i;\nfi; done\n\nThe results: no branch or tree appears to contain that blob.\n\nSo I tried a different approach:\n\nfor i in $( grep tree gvp.out  | awk '{ print $1 }' ); do if git\nls-tree $i | grep b32c3d8e8e24d8d3035cf52f606c2873315fe2b8 >\n/dev/null; then echo $i; fi ; done\n\nThis time, I find (at least) one tree\n(d813af1537358496ca34958bbff08b87590607bf) with the blob.\nBut which branches might that tree appear in? None.\n\nFor each branch, I ran \"git ls-tree -l -r -t\" and saved the output in\na file (one per branch).\nThen I grepped each file for the tree (\n(d813af1537358496ca34958bbff08b87590607bf) - no luck.\nI grepped each file for the blob (b32...) - no luck.\n\nThe results seem to suggest that I have packed trees which reference\nblobs, but that the trees themselves are not referenced in any branch\nand therefore I would expect that they would be pruned.\n\n\n-- \nJon\n"},{"id":"133720","messageId":"alpine.LFD.2.00.1002051539080.1681@xanadu.home","threadId":"22534","inReplyTo":"cccedfc61002051145q1ff673e7s3db3bd7290be25e1@mail.gmail.com","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-02-05T20:51:07Z","receivedAt":"2010-02-05T20:51:07Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 5 Feb 2010, Jon Nelson wrote:\n\n> [Using git 1.6.4.2]\n> \n> In one repo I have (136G objects directory, fully packed) I'm having\n> some trouble.\n> I've run git-gc --prune=now, git repack -Adf, and so on a half-dozen\n> times and each time I do so it gets bigger, not smaller.\n\nPlease tell us more.\n\n> Setting that aside for the moment, however, I've run into a stranger problem.\n> \n> So I use \"git verify-pack -v > gvp.out\" and \"sort -k3nr < gvp.out |\n> head -n 20\" to find the top 20 largest blobs.\n> So I have a blob, b32c3d8e8e24d8d3035cf52f606c2873315fe2b8, and now I\n> want to know what tree (or trees) it is in, so I try this:\n> \n> \n> for i in $( git branch -a | sed -e 's/\\*//g' | grep -v branch ); do if\n> git ls-tree -l -r -t $i | grep\n> b32c3d8e8e24d8d3035cf52f606c2873315fe2b8 > /dev/null; then echo $i;\n> fi; done\n> \n> The results: no branch or tree appears to contain that blob.\n\nWhat you did above is simply to list trees that are reachable from the \n_heads_ of your branches.  If the blob belongs to a commit which isn't \nthe latest revision of any of your branches then you won't see it like \nthat.\n\n> So I tried a different approach:\n> \n> for i in $( grep tree gvp.out  | awk '{ print $1 }' ); do if git\n> ls-tree $i | grep b32c3d8e8e24d8d3035cf52f606c2873315fe2b8 >\n> /dev/null; then echo $i; fi ; done\n> \n> This time, I find (at least) one tree\n> (d813af1537358496ca34958bbff08b87590607bf) with the blob.\n> But which branches might that tree appear in? None.\n> \n> For each branch, I ran \"git ls-tree -l -r -t\" and saved the output in\n> a file (one per branch).\n> Then I grepped each file for the tree (\n> (d813af1537358496ca34958bbff08b87590607bf) - no luck.\n> I grepped each file for the blob (b32...) - no luck.\n> \n> The results seem to suggest that I have packed trees which reference\n> blobs, but that the trees themselves are not referenced in any branch\n> and therefore I would expect that they would be pruned.\n\nNO.  If those trees and blobs are stil there then they do get \nreferenced.  But not from the latest commit on any of your branches.  \nYou need to dig further down in history to find a commit that actually \nreferences that blob/tree.  One easy method is to do:\n\n\tgit log --raw --all\n\nand within the pager ('less' by default) simply search for \"b32c3d8\".\n\n\nNicolas\n"},{"id":"133725","messageId":"cccedfc61002051304t6030d3f7if4bb14709ee6c918@mail.gmail.com","threadId":"22534","inReplyTo":"alpine.LFD.2.00.1002051539080.1681@xanadu.home","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Jon Nelson","fromEmail":"jnelson@jamponi.net","sentAt":"2010-02-05T21:04:13Z","receivedAt":"2010-02-05T21:04:13Z","isPatch":false,"sender":{"key":"jnelson@jamponi.net","avatar":null},"body":"On Fri, Feb 5, 2010 at 2:51 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n> On Fri, 5 Feb 2010, Jon Nelson wrote:\n>\n>> [Using git 1.6.4.2]\n>>\n>> In one repo I have (136G objects directory, fully packed) I'm having\n>> some trouble.\n>> I've run git-gc --prune=now, git repack -Adf, and so on a half-dozen\n>> times and each time I do so it gets bigger, not smaller.\n>\n> Please tell us more.\n\nI'll tell you whatever I can -- as soon as I know what it is you want.\n\n>> Setting that aside for the moment, however, I've run into a stranger problem.\n>>\n>> So I use \"git verify-pack -v > gvp.out\" and \"sort -k3nr < gvp.out |\n>> head -n 20\" to find the top 20 largest blobs.\n>> So I have a blob, b32c3d8e8e24d8d3035cf52f606c2873315fe2b8, and now I\n>> want to know what tree (or trees) it is in, so I try this:\n>>\n>>\n>> for i in $( git branch -a | sed -e 's/\\*//g' | grep -v branch ); do if\n>> git ls-tree -l -r -t $i | grep\n>> b32c3d8e8e24d8d3035cf52f606c2873315fe2b8 > /dev/null; then echo $i;\n>> fi; done\n>>\n>> The results: no branch or tree appears to contain that blob.\n>\n> What you did above is simply to list trees that are reachable from the\n> _heads_ of your branches.  If the blob belongs to a commit which isn't\n> the latest revision of any of your branches then you won't see it like\n> that.\n>\n>> So I tried a different approach:\n>>\n>> for i in $( grep tree gvp.out  | awk '{ print $1 }' ); do if git\n>> ls-tree $i | grep b32c3d8e8e24d8d3035cf52f606c2873315fe2b8 >\n>> /dev/null; then echo $i; fi ; done\n>>\n>> This time, I find (at least) one tree\n>> (d813af1537358496ca34958bbff08b87590607bf) with the blob.\n>> But which branches might that tree appear in? None.\n>>\n>> For each branch, I ran \"git ls-tree -l -r -t\" and saved the output in\n>> a file (one per branch).\n>> Then I grepped each file for the tree (\n>> (d813af1537358496ca34958bbff08b87590607bf) - no luck.\n>> I grepped each file for the blob (b32...) - no luck.\n>>\n>> The results seem to suggest that I have packed trees which reference\n>> blobs, but that the trees themselves are not referenced in any branch\n>> and therefore I would expect that they would be pruned.\n>\n> NO.  If those trees and blobs are stil there then they do get\n> referenced.  But not from the latest commit on any of your branches.\n> You need to dig further down in history to find a commit that actually\n> references that blob/tree.  One easy method is to do:\n>\n>        git log --raw --all\n>\n> and within the pager ('less' by default) simply search for \"b32c3d8\".\n\nOK. I'm piping \"git log --raw --all\" to a file this very moment. It'll\ntake a while. However, one thing I did not mention is that there\n*should* be a 1:1 correlation between branches and commits. As in,\nevery time I did a commit, the commit was on a new branch. I'll look\ninto this, as I've fiddled with the repo a bunch of different ways\nlately. I suspect the answer will be found in the logs.\n\nThanks for the response!\n\n-- \nJon\n"},{"id":"133733","messageId":"alpine.LFD.2.00.1002051634380.1681@xanadu.home","threadId":"22534","inReplyTo":"cccedfc61002051304t6030d3f7if4bb14709ee6c918@mail.gmail.com","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-02-05T21:45:15Z","receivedAt":"2010-02-05T21:45:15Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Fri, 5 Feb 2010, Jon Nelson wrote:\n\n> On Fri, Feb 5, 2010 at 2:51 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n> > On Fri, 5 Feb 2010, Jon Nelson wrote:\n> >\n> >> [Using git 1.6.4.2]\n> >>\n> >> In one repo I have (136G objects directory, fully packed) I'm having\n> >> some trouble.\n> >> I've run git-gc --prune=now, git repack -Adf, and so on a half-dozen\n> >> times and each time I do so it gets bigger, not smaller.\n> >\n> > Please tell us more.\n> \n> I'll tell you whatever I can -- as soon as I know what it is you want.\n\nI want to know what you did and what results you obtained to lead you to \nbelieve that 'git-gc --prune=now' makes your repository bigger.\n\n> > You need to dig further down in history to find a commit that actually\n> > references that blob/tree.  One easy method is to do:\n> >\n> >        git log --raw --all\n> >\n> > and within the pager ('less' by default) simply search for \"b32c3d8\".\n> \n> OK. I'm piping \"git log --raw --all\" to a file this very moment. It'll\n> take a while. However, one thing I did not mention is that there\n> *should* be a 1:1 correlation between branches and commits. As in,\n> every time I did a commit, the commit was on a new branch. \n\nOuch.  Why?  I'm curious.\n\n> I'll look into this, as I've fiddled with the repo a bunch of \n> different ways lately. I suspect the answer will be found in the logs.\n\nIf the blob you're looking for is not in that log, then it must be \nreferenced by one of the reflog entries.  The reflog is a log of all the \ncommits your branches, and the HEAD pointer, ever pointed at.  Those \nentries are kept for a certain period (3 months by default).  The HEAD \nreflog will keep a reference of all branches you might have visited, \neven deleted ones.  So another log:\n\n\tgit log --raw -g HEAD\n\nshould ultimately give you the blob reference.\n\n\nNicolas\n"},{"id":"133801","messageId":"cccedfc61002060553w464d5d9drd582b3b3c3f35227@mail.gmail.com","threadId":"22534","inReplyTo":"alpine.LFD.2.00.1002051634380.1681@xanadu.home","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Jon Nelson","fromEmail":"jnelson@jamponi.net","sentAt":"2010-02-06T13:53:13Z","receivedAt":"2010-02-06T13:53:13Z","isPatch":false,"sender":{"key":"jnelson@jamponi.net","avatar":null},"body":"On Fri, Feb 5, 2010 at 3:45 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n> On Fri, 5 Feb 2010, Jon Nelson wrote:\n>\n>> On Fri, Feb 5, 2010 at 2:51 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n>> > On Fri, 5 Feb 2010, Jon Nelson wrote:\n>> >\n>> >> [Using git 1.6.4.2]\n>> >>\n>> >> In one repo I have (136G objects directory, fully packed) I'm having\n>> >> some trouble.\n>> >> I've run git-gc --prune=now, git repack -Adf, and so on a half-dozen\n>> >> times and each time I do so it gets bigger, not smaller.\n>> >\n>> > Please tell us more.\n>>\n>> I'll tell you whatever I can -- as soon as I know what it is you want.\n>\n> I want to know what you did and what results you obtained to lead you to\n> believe that 'git-gc --prune=now' makes your repository bigger.\n\nLast night, the repo size was 153G after removing some commits and\nobjects by way of git filter-branch.\nI'm using \"du -sh\" in the .git directory to determine the disk usage.\n\nBefore: 136G\ngit repack -dAl\nAfter: 153G\n\nThen, just to make sure of some things, I changed nothing and simply\nre-ran \"git repack -dAl\".\nAfter: 167G\n\nConfig:\n\n[core]\n        repositoryformatversion = 0\n        filemode = true\n        bare = true\n        logallrefupdates = true\n        compression = 0\n        loosecompression = 0\n        packedGitLimit = 1500M\n        #packedGitWindowSize = 32m\n[pack]\n        packsizelimit = 256m\n        windowMemory = 128m\n        deltaCacheSize = 1m\n[gc]\n    threads = 1\n    packrefs = true\n\n[repack]\n        usedeltabaseoffset = true\n\nIncorporating the system and global config (git config -l):\n\npack.packsizelimit=2M\npack.threads=1\ngc.auto=0\ngc.autopacklimit=0\ncore.repositoryformatversion=0\ncore.filemode=true\ncore.bare=true\ncore.logallrefupdates=true\ncore.compression=0\ncore.loosecompression=0\ncore.packedgitlimit=1500M\npack.packsizelimit=256m\npack.windowmemory=128m\npack.deltacachesize=1m\ngc.threads=1\ngc.packrefs=true\nrepack.usedeltabaseoffset=true\n\n-- \nJon\n"},{"id":"133839","messageId":"alpine.LFD.2.00.1002061935180.1681@xanadu.home","threadId":"22534","inReplyTo":"cccedfc61002060553w464d5d9drd582b3b3c3f35227@mail.gmail.com","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-02-07T01:16:58Z","receivedAt":"2010-02-07T01:16:58Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sat, 6 Feb 2010, Jon Nelson wrote:\n\n> Last night, the repo size was 153G after removing some commits and\n> objects by way of git filter-branch.\n> I'm using \"du -sh\" in the .git directory to determine the disk usage.\n> \n> Before: 136G\n> git repack -dAl\n> After: 153G\n\nWhy are you using -A instead of -a ?\n\n> Then, just to make sure of some things, I changed nothing and simply\n> re-ran \"git repack -dAl\".\n> After: 167G\n\nCould you run 'git count-objects -v' before and after a repack in such \ncases as well?\n\n> [pack]\n>         packsizelimit = 256m\n\nWhy are you using this?\n\n> pack.packsizelimit=2M\n\nThis is even worse.  150G / 2M per pack = approx 75000 packs.\n\nWhat you should be aiming for is a _single_ pack for best performances.  \nA couple packs is common and doesn't make much of a difference.  A \nhundred packs is getting a bit large.  But 75000 packs is totally \ninsane.\n\n\nNicolas\n"},{"id":"133875","messageId":"cccedfc61002070948m2491b1e2ua633f125f3573ff9@mail.gmail.com","threadId":"22534","inReplyTo":"alpine.LFD.2.00.1002061935180.1681@xanadu.home","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Jon Nelson","fromEmail":"jnelson@jamponi.net","sentAt":"2010-02-07T17:48:19Z","receivedAt":"2010-02-07T17:48:19Z","isPatch":false,"sender":{"key":"jnelson@jamponi.net","avatar":null},"body":"On Sat, Feb 6, 2010 at 7:16 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n> On Sat, 6 Feb 2010, Jon Nelson wrote:\n>\n>> Last night, the repo size was 153G after removing some commits and\n>> objects by way of git filter-branch.\n>> I'm using \"du -sh\" in the .git directory to determine the disk usage.\n>>\n>> Before: 136G\n>> git repack -dAl\n>> After: 153G\n>\n> Why are you using -A instead of -a ?\n\nAs it turns out, I've been using both -a and -A. I suspect -A is a\ntypo on my part.\n\n>> Then, just to make sure of some things, I changed nothing and simply\n>> re-ran \"git repack -dAl\".\n>> After: 167G\n>\n> Could you run 'git count-objects -v' before and after a repack in such\n> cases as well?\n\nYes.\n\n>> [pack]\n>>         packsizelimit = 256m\n>\n> Why are you using this?\n\nI didn't want my pack files to be too huge. I've bumped that up to 2G.\n\n>> pack.packsizelimit=2M\n\nMy ~/.gitconfig normally uses 2M for easy rsyncing. In this repo I\nthought the value was overridden by the project's config (which was\nspecifying 256m and now specifies 2048m).\n\nSuboptimal or not, it still doesn't explain why the repo grows with each repack.\n\nNow running:\n\n git count-objects -v ; git repack -ad ; git count-objects -v\n\n\n-- \nJon\n"},{"id":"133899","messageId":"cccedfc61002071540l7019d5d0mc4666dbd81189064@mail.gmail.com","threadId":"22534","inReplyTo":"cccedfc61002070948m2491b1e2ua633f125f3573ff9@mail.gmail.com","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Jon Nelson","fromEmail":"jnelson@jamponi.net","sentAt":"2010-02-07T23:40:17Z","receivedAt":"2010-02-07T23:40:17Z","isPatch":false,"sender":{"key":"jnelson@jamponi.net","avatar":null},"body":"On Sun, Feb 7, 2010 at 11:48 AM, Jon Nelson <jnelson@jamponi.net> wrote:\n> On Sat, Feb 6, 2010 at 7:16 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n>> On Sat, 6 Feb 2010, Jon Nelson wrote:\n>>\n>>> Last night, the repo size was 153G after removing some commits and\n>>> objects by way of git filter-branch.\n>>> I'm using \"du -sh\" in the .git directory to determine the disk usage.\n>>>\n>>> Before: 136G\n>>> git repack -dAl\n>>> After: 153G\n>>\n>> Why are you using -A instead of -a ?\n>\n> As it turns out, I've been using both -a and -A. I suspect -A is a\n> typo on my part.\n>\n>>> Then, just to make sure of some things, I changed nothing and simply\n>>> re-ran \"git repack -dAl\".\n>>> After: 167G\n>>\n>> Could you run 'git count-objects -v' before and after a repack in such\n>> cases as well?\n>\n> Yes.\n>\n>>> [pack]\n>>>         packsizelimit = 256m\n>>\n>> Why are you using this?\n>\n> I didn't want my pack files to be too huge. I've bumped that up to 2G.\n>\n>>> pack.packsizelimit=2M\n>\n> My ~/.gitconfig normally uses 2M for easy rsyncing. In this repo I\n> thought the value was overridden by the project's config (which was\n> specifying 256m and now specifies 2048m).\n>\n> Suboptimal or not, it still doesn't explain why the repo grows with each repack.\n>\n> Now running:\n>\n>  git count-objects -v ; git repack -ad ; git count-objects -v\n\nThe results (two lines of whitespace added for clarity):\n\nturnip:/mnt/backups.git # git count-objects -v ; git repack -ad ; git\ncount-objects -v\ncount: 0\nsize: 0\nin-pack: 2408195\npacks: 675\nsize-pack: 174924053\nprune-packable: 0\ngarbage: 0\n\nCounting objects: 266309, done.\nCompressing objects: 100% (207820/207820), done.\nWriting objects: 100% (266309/266309), done.\nTotal 266309 (delta 47751), reused 227730 (delta 45117)\n\ncount: 0\nsize: 0\nin-pack: 2674504\npacks: 686\nsize-pack: 196517094\nprune-packable: 0\ngarbage: 0\nturnip:/mnt/backups.git # du -sh .\n188G    .\nturnip:/mnt/backups.git # find objects -name '*.pack' | wc -l\n686\nturnip:/mnt/backups.git #\n\nSo we went from 136G to 153G to 167G and now 188G.\n\nThe oldest .pack file is Jan 29 at 7.0M\nThe pack files vary considerably in size from 220B to 2.0G\nThere are 433 packs that are between 200M and 300M in size.\nThere are 78 packs that are between 500M and 600M in size.\nThere are 211 packs that are less than 256M in size.\n\nI ran \"git prune -v --prune=now\" and it claims to have removed nothing\nand the exit status was 0.\n\nI hope this helps!\n\n-- \nJon\n"},{"id":"133906","messageId":"alpine.LFD.2.00.1002071937000.1681@xanadu.home","threadId":"22534","inReplyTo":"cccedfc61002071540l7019d5d0mc4666dbd81189064@mail.gmail.com","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Nicolas Pitre","fromEmail":"nico@fluxnic.net","sentAt":"2010-02-08T02:11:36Z","receivedAt":"2010-02-08T02:11:36Z","isPatch":false,"sender":{"key":"nico@fluxnic.net","avatar":"https://avatars.githubusercontent.com/u/702790?v=4"},"body":"On Sun, 7 Feb 2010, Jon Nelson wrote:\n> On Sat, Feb 6, 2010 at 7:16 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n> >On Sat, 6 Feb 2010, Jon Nelson wrote:\n> >>         packsizelimit = 256m\n> >\n> > Why are you using this?\n> \n> I didn't want my pack files to be too huge. I've bumped that up to 2G.\n> \n> >> pack.packsizelimit=2M\n> \n> My ~/.gitconfig normally uses 2M for easy rsyncing. In this repo I\n> thought the value was overridden by the project's config (which was\n> specifying 256m and now specifies 2048m).\n\nWhy do you synchronize with rsync?  Why not a simple 'git fetch' which \nis likely to be much faster in any case?\n\n> Suboptimal or not, it still doesn't explain why the repo grows with each repack.\n\nThat's strange indeed.  Except that ...\n\n> turnip:/mnt/backups.git # git count-objects -v ; git repack -ad ; git\n> count-objects -v\n> count: 0\n> size: 0\n> in-pack: 2408195\n> packs: 675\n> size-pack: 174924053\n> prune-packable: 0\n> garbage: 0\n> \n> Counting objects: 266309, done.\n> Compressing objects: 100% (207820/207820), done.\n> Writing objects: 100% (266309/266309), done.\n> Total 266309 (delta 47751), reused 227730 (delta 45117)\n> \n> count: 0\n> size: 0\n> in-pack: 2674504\n> packs: 686\n> size-pack: 196517094\n> prune-packable: 0\n> garbage: 0\n\nYou previously had 2408195 packed objects and after the repack this is \n2674504.  The difference is 266309, which incidentally is the number of \nobjects 'git repack' counted.\n\nIf that 266309 object count is stable between repack attempts (and it \nshould), then something is failing to clean up the old packs.  If I \nremember right, you were playing with modifications to git-repack.sh \nlately?  It is well possible that you broke it.\n\nI tried to simulate your setup but I'm regretting it now.  My test repo \nexpanded to 10x its original size with 900 packs.  Git is also painfully \nslow now while I'm trying to repack it properly back into a single pack.  \nHowever I wasn't able to reproduce your repository growth.  Please \nretest with a vanilla Git version.\n\nI think you should really consider using an unlimited pack size.  Not \nonly you'll save quite a lot of disk space, but Git will be much faster \ntoo.  And Git itself is far more efficient than rsync at keeping Git \nrepositories in sync, regardless of pack sizes (unless you have too many \npacks of course).\n\n\nNicolas\n"},{"id":"133930","messageId":"cccedfc61002080912ybc7775by24fd5511fb67eac5@mail.gmail.com","threadId":"22534","inReplyTo":"alpine.LFD.2.00.1002071937000.1681@xanadu.home","subject":"Re: git gc / git repack not removing unused objects?","fromName":"Jon Nelson","fromEmail":"jnelson@jamponi.net","sentAt":"2010-02-08T17:12:03Z","receivedAt":"2010-02-08T17:12:03Z","isPatch":false,"sender":{"key":"jnelson@jamponi.net","avatar":null},"body":"On Sun, Feb 7, 2010 at 8:11 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n> On Sun, 7 Feb 2010, Jon Nelson wrote:\n>> On Sat, Feb 6, 2010 at 7:16 PM, Nicolas Pitre <nico@fluxnic.net> wrote:\n>> >On Sat, 6 Feb 2010, Jon Nelson wrote:\n>> >>         packsizelimit = 256m\n>> >\n>> > Why are you using this?\n>>\n>> I didn't want my pack files to be too huge. I've bumped that up to 2G.\n>>\n>> >> pack.packsizelimit=2M\n>>\n>> My ~/.gitconfig normally uses 2M for easy rsyncing. In this repo I\n>> thought the value was overridden by the project's config (which was\n>> specifying 256m and now specifies 2048m).\n>\n> Why do you synchronize with rsync?  Why not a simple 'git fetch' which\n> is likely to be much faster in any case?\n\nOften I need to sync the entire config, and am frequently rsyncing\n(although not in this case) rsyncing repos made with git-svn. I've had\ntrouble using git clone on these repos, as I really want the duplicate\nrepo to use the svn upstream and not the git repo it was cloned from.\n\n..\n\n> You previously had 2408195 packed objects and after the repack this is\n> 2674504.  The difference is 266309, which incidentally is the number of\n> objects 'git repack' counted.\n>\n> If that 266309 object count is stable between repack attempts (and it\n> should), then something is failing to clean up the old packs.  If I\n> remember right, you were playing with modifications to git-repack.sh\n> lately?  It is well possible that you broke it.\n\nIndeedly do I did.\n\nThe (now expected) results:\n\nturnip:/mnt/backups.git # du -sh ; echo ; git count-objects -v ; echo\n; git repack -ad ; echo ; git count-objects -v ; echo ; du -sh\n204G    .\n\ncount: 0\nsize: 0\nin-pack: 2939744\npacks: 694\nsize-pack: 213284091\nprune-packable: 0\ngarbage: 0\n\nCounting objects: 266309, done.\nCompressing objects: 100% (205176/205176), done.\nWriting objects: 100% (266309/266309), done.\nTotal 266309 (delta 47781), reused 230413 (delta 47770)\n\ncount: 0\nsize: 0\nin-pack: 266309\npacks: 11\nsize-pack: 21590451\nprune-packable: 0\ngarbage: 0\n\n21G     .\nturnip:/mnt/backups.git #\n\nand of course now everything is nice and (reasonably) fast.\nThanks for the help, and pointing out that the problem was .. me.\n\n-- \nJon\n"}]}