Re: Performance issue exposed by git-filter-branch

From: Thomas Rast <trast@student.ethz.ch>
Date: Dec 17, 2010, 01:54 UTC
Message-ID: <201012170254.13005.trast@student.ethz.ch>
In-Reply-To: <41C1B4AC-8427-4D62-BEB6-689A4BE4EE5B@irridia.com>

Ken Brownfield wrote:

> git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch -- bigdirtree stuff/a stuff/b stuff/c stuff/dir/{a,b,c}' --prune-empty --tag-name-filter cat -- --all

[...]

> Now that the same repository has grown, this same filter-branch
> process now takes 6.5 *days* at 100% CPU on the same machine (2x4
> Xeon, x86_64) on git-1.7.3.2.  There's no I/O, memory, or other
> resource contention.

If all you do is an index-filter for deletion, I think it should be rather easy to achieve good results by filtering the fast-export stream to remove these files, and then piping that back to fast-import.

(It's just that AFAIK nobody has written that code yet.)

-- 
Thomas Rast
trast@{inf,student}.ethz.ch