Re: [PATCH 1/3] Add --blob-filter option to filter-branch.

> From: Jeff King <peff@peff.net>
> 
> On Tue, Apr 22, 2008 at 12:51:14PM -0400, Avery Pennarun wrote:
> 
> > Do you think git would benefit from having a generalized version of
> > this script?  Basically, the user provides a "munge" script on the
> > command line, and there's a git-filter-branch mode for auto-munging
> > (with a cache) every file in every checkin.  Even if it's *only* ever
> > used for CRLF, I can imagine this being useful to a lot of people.
> 
> It was easy enough to work up the patch below, which allows
> 
>   git filter-branch --blob-filter 'tr a-z A-Z'
> 
> However, it's _still_ horribly slow. Shell script is nice and flexible,
> but running a tight loop like this is just painful. I suspect
> filter-branch in something like perl would be a lot faster and just as
> flexible (you could even do it in C, but you'd probably have to invent a
> little domain-specific scripting language).
> 
> It is still much better performance than a tree filter, though:
> 
>   $ cd git && time git filter-branch --tree-filter '
>       find . -type f | while read f; do
>         tr a-z A-Z <"$f" >tmp
>         mv tmp "$f"
>       done
>     ' HEAD~10..HEAD
> 
>   real    4m38.626s
>   user    1m32.726s
>   sys     2m51.163s
> 
>   $ cd git && git filter-branch --blob-filter 'tr a-z A-Z' HEAD~10..HEAD
>   real    1m40.809s
>   user    0m36.822s
>   sys     1m14.273s
> 
> Lots of system time in both. I'm sure we spend a fair bit of time
> hitting our very large map and blob-cache directories, which would be
> much more nicely implemented as associative arrays in memory (if we were
> using a more featureful language).
> 
> Anyway, here is the patch. I don't know if it is even worth applying,
> since it is still painfully slow.