Re: Blobs not referenced by file (anymore) are not removed by GC

From: Jeff King <peff@peff.net>
Date: Dec 9, 2014, 14:14 UTC
Message-ID: <20141209141457.GA18544@peff.net>
In-Reply-To: <5485D03F.3060008@fu-berlin.de>

On Mon, Dec 08, 2014 at 05:22:23PM +0100, Martin Scherer wrote:

Show 8 quoted lines

> # invoke bfg --delete-folders something multiple times with different
> pattern.
> 
> # try to cleanup
> 
> git gc --aggressive --prune=now # big blobs still in history
> git fsck # no results
> git fsck --full  --unreachable --dangling # no results

Might you still have reflogs pointing to the objects? Try:

  git reflog expire --expire-unreachable=now --all

I also don't know if BFG keeps backup refs around (filter-branch, for example, writes a copy of the original refs into refs/original; you would want to delete that if you're trying to slim down the repo).

In general, you can see the on-disk size of the objects required for a particular ref with something like:

  size() {
    git rev-list --objects "$@" |
    cut -d' ' -f1 |
    git cat-file --batch-check='%(objectsize:disk)' |
    perl -lne '$t += $_; END { print $t }'
  }

  # size of master branch
  size master

  # size of each ref on top of what is in the master branch
  git for-each-ref --format='%(refname)' |
  while read ref; do
    echo "$(size master..$ref) $ref"
  done | sort -rn

Note that these sizes are somewhat approximate. We may store object X needed by one ref as a delta against Y used by another ref. The accounting shows X as tiny compared to Y. And then a repack may find the delta in the opposite direction. But if you're talking about rewriting history to drop a bunch of gigantic objects, the output of the final loop is a good way to see which refs are still referring to the old history.

-Peff