{"thread":{"id":"47788","subject":"How to produce a loose ref+size explosion via pruning + git-gc","startedAt":"2018-02-08T15:37:43Z","lastAt":"2018-02-08T15:37:43Z","messageCount":1,"participants":["Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"338704","messageId":"87fu6bmr0j.fsf@evledraar.gmail.com","threadId":"47788","inReplyTo":null,"subject":"How to produce a loose ref+size explosion via pruning + git-gc","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2018-02-08T15:37:32Z","receivedAt":"2018-02-08T15:37:43Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"I'll probably submit docs for this eventually, but the docs in my\n--prune-tags series were already hard enough to review. Try running this\n\n    (\n        rm -rf /tmp/git &&\n        git clone https://github.com/git/git /tmp/git &&\n        cd /tmp/git >/dev/null &&\n        du -sh .git &&\n        git rev-list --all origin/master.. | wc -l &&\n        for clone in gitster peff avar chriscool mhagger pclouds Microsoft\n        do\n            git remote add $clone https://github.com/$clone/git &&\n            git fetch -q $clone\n        done &&\n        git gc &&\n        du -sh .git &&\n        git rev-list --all origin/master.. | wc -l &&\n        git fetch -q origin --prune 'refs/tags/*:refs/tags/*' &&\n        for remote in $(git remote | grep -v origin)\n        do\n            git remote rm $remote\n        done &&\n        git gc &&\n        du -sh .git &&\n        git rev-list --all origin/master.. | wc -l\n    )\n\nThe output is:\n\n    108M    .git\n    2222\n    160M    .git\n    62220\n    1.9G    .git\n    2222\n\nI.e. a fresh clone of git.git is 108MB, add a few more repos that have\ndiverged quite a bit in its network ad it's 160MB repacked.\n\nNow remove those remotes and \"git gc\" and it's 1.9GB, even though it's\ndivergent by the same 2222 commits from master as the 108MB, but after\nrunning:\n\n    git prune --expire=now\n\nIt becomes ~108MB again.\n\nNow this is all expected behavior, we've made a bunch of objects\nunreferenced, so they all get exploded into loose objects, which takes a\nlot of space.\n\nIt's an interesting caveat when setting fetch.prune=true on checkouts\nthat didn't previously have it and might have lots of brances to be\npruned.\n\nFor reasons I won't go into I'd had that disabled for a while here at\nwork, and after re-enabling it we had some repos whose .git is usually\n2.5G explode to 30G once git-gc ran.\n\nThe workaround is to set gc.pruneExpire low enough that when the gc hits\nall those objects get deleted, I set it to 1 day (from the default of 2\nweeks).\n\nBut it doesn't help with repos that have already run git-gc and exploded\nin size, much to the confusion of users on those systems, those need a\nmanual git-prune.\n\nPotential solutions to this have been discussed ad-nauseam here on\nlist. Let's not go into that (unless someone feels like it).\n\nI mainly wanted to send this for later reference, and have some\nsearchable record in case someone's confused when they turn on prune and\ntheir repo increases to 10x the previous size.\n"}]}