Re: filter-branch IO optimization

From: Thomas Rast <trast@student.ethz.ch>
Date: Oct 11, 2012, 20:34 UTC
Message-ID: <878vbc21f3.fsf@pctrast.inf.ethz.ch>
In-Reply-To: <fa1e05a5-54b3-47ff-bd28-dc463ebbc4bd@zcs>

Enrico Weigelt <enrico.weigelt@vnc.biz> writes:

> for certain projects, I need to regularily run filter-branch on quite
> large repos (>10k commits), and that needs to be run multiple times,
> which takes several hours, so I'm looking for optimizations.

[...]

> #2: run a tree-filter which:
>     * removes all files not belonging to the wanted module
>     * move the module directory under another subdir (./addons/)
>     * fix author/comitter name/email if empty (because otherwise fails)

The usual advice is "use an index-filter instead". It's *much* faster than a tree filter. However:

>     * fix charater sets and indentions of source files

That last step is rather crazy. At the very least you will want to only operate on files that were changed since the parent commit, so as to avoid scanning the whole tree. If you do this right, it should also fit into an index-filter.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch