git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Blobs not referenced by file (anymore) are not removed by GC

From
Roberto Tyley <roberto.tyley@gmail.com>
Date
Dec 8, 2014, 16:47 UTC
Message-ID
<CAFY1edarWL1B72ZO0m=RALJ+p3KtnBBX9qQ85iueU6EVD0akaA@mail.gmail.com>
In-Reply-To
<CAFY1edaEq1zYV0vgSfiPAXU6bqVBzaA-apVnSn8DBMbzcAa2tQ@mail.gmail.com>

Hi Martin, I'm the developer of the BFG - I'd guess that there probably isn't a bug for Git developers here, so you might want to open one or more issues at https://github.com/rtyley/bfg-repo-cleaner/issues, where I'd be happy to take a look.

best regards, Roberto

Show 56 quoted lines
> On 8 Dec 2014 16:35, "Martin Scherer" <m.scherer@fu-berlin.de> wrote:
>>
>> Hi,
>>
>> after using BFG on a repo given certain directory globs, all of those
>> files(names) are gone from history, but can not be collected by garbage
>> collection anymore. So the blobs of the underlying files are not deleted
>> and only the file names are not associated with the blob anymore. I
>> wonder, if I discovered a bug (at least in bfg). But I expect git to
>> discover that this blobs are not used in any way (so they have to
>> associated to something right?)
>>
>> # invoke bfg --delete-folders something multiple times with different
>> pattern.
>>
>> # try to cleanup
>>
>> git gc --aggressive --prune=now # big blobs still in history
>> git fsck # no results
>> git fsck --full  --unreachable --dangling # no results
>>
>> to verify if the blobs are still there, see the output of
>>
>> git gc && git verify-pack -v .git/objects/pack/pack-*.idx | egrep "^\w+
>> blob\W+[0-9]+ [0-9]+ [0-9]+$" | sort -k 3 -n -r > bigobjects
>> .txt
>>
>> head bigobjects.txt # outputs 9451427d7335395779b91864418630d2f0af780a
>> blob   7895212 1869047 7657491
>>
>>
>> Also if bfg is being told to remove the biggest blob (bfg -B 1) with
>> no-blob-protection, it does not succeed in removing it.
>>
>> --- output of bfg -B 1
>>
>> Found 1 blob ids for large blobs - biggest=7895212 smallest=7895212
>> ....
>>
>> BFG aborting: No refs to update - no dirty commits found??
>> ---
>>
>> The repo can be found here.
>>
>> https://github.com/marscher/stallone_stale_objects
>>
>> I will restart all over to cleanup the history, but I guess this might
>> be interesting for git developers.
>>
>>
>> Best,
>> Martin
>> --
>> To unsubscribe from this list: send the line "unsubscribe git" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
Previous: Roberto Tyley
Message 9 of 9 in “Blobs not referenced by file (anymore) are not removed by GC”
  1. Martin SchererDec 8, 2014
  2. Jeff KingDec 9, 2014
  3. Roberto TyleyDec 9, 2014
  4. Jeff KingDec 9, 2014
  5. Roberto TyleyDec 9, 2014
  6. Jeff KingDec 10, 2014
  7. Junio C HamanoDec 10, 2014
  8. Roberto TyleyDec 10, 2014
  9. Roberto TyleyDec 8, 2014

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.