git/list[1] front-page[2] threads[3] people[4] search[5] about
 

Re: git filter-branch should run git gc --auto

From
Kevin Ballard <kevin@sb.org>
Date
Jan 23, 2008, 03:03 UTC
Message-ID
<175DB3F7-2E39-4450-945F-E33D83EF2792@sb.org>
In-Reply-To
<7vwsq1p82i.fsf@gitster.siamese.dyndns.org>
On Jan 22, 2008, at 9:52 PM, Junio C Hamano wrote:
Show 45 quoted lines
> Junio C Hamano <gitster@pobox.com> writes:
>
>> Kevin Ballard <kevin@sb.org> writes:
>>
>>> Am I missing something, or does git-filter-branch really not do any
>>> garbage collection? I tried reading the source, but complex bash
>>> scripts are almost as bad as perl in terms of readability.
>>
>> Theoretically yes, and it largely depends on what you do, but
>> filter-branch goes over the objects that already exists in your
>> repository, and hopefully you won't be rewriting majority of
>> them.
>>
>> So the impact of not repacking is probably much less painful in
>> practice.
>>
>> But again as I said, it largely depends on what you do in your
>> filter.  If you are upcasing (or convert to NFD ;-)) the
>> contents of all of your blob objects, you would certainly want
>> to repack every once in a while.
>
> Something like this, perhaps?
>
> git-filter-branch.sh |    6 ++++++
> 1 files changed, 6 insertions(+), 0 deletions(-)
>
> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
> index ebf05ca..8e44001 100755
> --- a/git-filter-branch.sh
> +++ b/git-filter-branch.sh
> @@ -299,6 +299,12 @@ while read commit parents; do
> 			die "msg filter failed: $filter_msg"
> 	sh -c "$filter_commit" "git commit-tree" \
> 		$(git write-tree) $parentstr < ../message > ../map/$commit
> +
> +	if test $(( $i % 512 )) = 0
> +	then
> +		git gc --auto
> +	fi
> +
> done <../revs
>
> # In case of a subdirectory filter, it is possible that a specified  
> head
>

Offhand that looks good, but we'd probably want to unilaterally do another git-gc when we're done.

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index ebf05ca..32274a6 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -299,8 +299,16 @@ while read commit parents; do
  			die "msg filter failed: $filter_msg"
  	sh -c "$filter_commit" "git commit-tree" \
  		$(git write-tree) $parentstr < ../message > ../map/$commit
+
+	if test $(( $i % 512 )) = 0
+	then
+		git gc --auto
+	fi
+
  done <../revs

+git gc --auto
+
  # In case of a subdirectory filter, it is possible that a specified  
head
  # is not in the set of rewritten commits, because it was pruned by the
  # revision walker.  Fix it by mapping these heads to the next  
rewritten
-- 
Kevin Ballard
http://kevin.sb.org
kevin@sb.org
http://www.tildesoft.com
Previous: Junio C HamanoNext: Harvey Harrison
Message 18 of 27 in “git-svn should default to --repack”
  1. Kevin BallardJan 18, 2008
  2. Karl HasselströmJan 18, 2008
  3. Junio C HamanoJan 18, 2008
  4. Karl HasselströmJan 19, 2008
  5. Kevin BallardJan 19, 2008
  6. Let "git svn" run "git gc --auto" occasionallyKarl Hasselström, Jan 19, 2008
  7. Harvey HarrisonJan 19, 2008
  8. Eric WongJan 20, 2008
  9. Karl HasselströmJan 20, 2008
  10. Junio C HamanoJan 20, 2008
  11. Eric WongJan 21, 2008
  12. Junio C HamanoJan 22, 2008
  13. Eric WongJan 22, 2008
  14. Junio C HamanoJan 22, 2008
  15. git filter-branch should run git gc --autoKevin Ballard, Jan 23, 2008
  16. Junio C HamanoJan 23, 2008
  17. Junio C HamanoJan 23, 2008
  18. Kevin BallardJan 23, 2008
  19. Harvey HarrisonJan 23, 2008
  20. Kevin BallardJan 23, 2008
  21. Sam VilainJan 23, 2008
  22. Kevin BallardJan 23, 2008
  23. Mike HommeyJan 23, 2008
  24. Johannes SchindelinJan 23, 2008
  25. Junio C HamanoJan 23, 2008
  26. 1/2 git-svn: Don't call git-repack anymoreKarl Hasselström, Jan 20, 2008
  27. 2/2 Let "git svn" run "git gc --auto" occasionallyKarl Hasselström, Jan 20, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.