git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: Trying to use git-filter-branch to compress history by removing large, obsolete binary files

From
Elijah Newren <newren@gmail.com>
Date
Oct 8, 2007, 00:09 UTC
Message-ID
<51419b2c0710071709s2f797df0u986447f5455f306d@mail.gmail.com>
In-Reply-To
<20071007234039.GH2765@steel.home>
On 10/7/07, Alex Riesen <raa.lkml@gmail.com> wrote:
> you missed something. Your example compresses to about 124k.

What version of git are you running? I reran all the steps to which you responded (repeated below for clarity) with git-1.5.3.3 and still get 11MB. Also, you must have different filesystem extents than me since an empty git repo takes 196k here[1], so I don't think any repo is going to get down to 124k.

My understanding of the steps you suggest would work:

# Make a small repo mkdir test cd test git init echo hi > there git add there git commit -m 'Small repo'

# Add a random 10M binary file dd if=/dev/urandom of=testme.txt count=10 bs=1M git add testme.txt git commit -m 'Add big binary file'

# Remove the 10M binary file git rm testme.txt git commit -m 'Remove big binary file'

# Compress the repo, see how big the repo is git gc --aggressive --prune du -ks . # 10548K du -ks .git # 10532K

# Try to rewrite history to remove the binary file git-filter-branch --tree-filter 'rm -f testme.txt' HEAD git reset --hard

# Try to recompress and clean up, then check the new size git gc --aggressive --prune du -ks . # 10580K !?!?!? du -ks .git # 10564K

# Do the stuff Alex suggests to trim the history rm -rf .git/refs/original/ vi .git/packed-refs # Use vi to remove the line referring to refs/original... git reflog expire --all git gc --aggressive --prune du -ks . # Still 10564K

Thanks, Elijah

[1] An empty git repo takes 196k for me, as can be seen by: $mkdir tmp $cd tmp $git init $du -hs . 196K .

Previous: Alex RiesenNext: Alex Riesen
Message 7 of 31 in “Trying to use git-filter-branch to compress history by removing large, obsolete binary files”
  1. Elijah NewrenOct 7, 2007
  2. Frank LichtenheldOct 7, 2007
  3. Elijah NewrenOct 7, 2007
  4. Alex RiesenOct 7, 2007
  5. Elijah NewrenOct 7, 2007
  6. Alex RiesenOct 7, 2007
  7. Elijah NewrenOct 8, 2007
  8. Alex RiesenOct 8, 2007
  9. Andreas EricssonOct 8, 2007
  10. Dmitry PotapovOct 7, 2007
  11. Elijah NewrenOct 8, 2007
  12. Dmitry PotapovOct 8, 2007
  13. Andreas EricssonOct 8, 2007
  14. Karl HasselströmOct 8, 2007
  15. Dmitry PotapovOct 8, 2007
  16. Karl HasselströmOct 8, 2007
  17. Johannes SchindelinOct 7, 2007
  18. Elijah NewrenOct 7, 2007
  19. Johannes SchindelinOct 7, 2007
  20. Elijah NewrenOct 7, 2007
  21. Johannes SchindelinOct 8, 2007
  22. Elijah NewrenOct 8, 2007
  23. Sam VilainOct 8, 2007
  24. J. Bruce FieldsOct 8, 2007
  25. Johannes SchindelinOct 8, 2007
  26. Johannes SixtOct 8, 2007
  27. J. Bruce FieldsOct 8, 2007
  28. Theodore TsoOct 8, 2007
  29. J. Bruce FieldsOct 8, 2007
  30. Johannes SchindelinOct 9, 2007
  31. Alex RiesenOct 7, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.