Converting repo from HG, `git filter-branch --prune-empty -- --all` is extremely slow and errors out.

From: John Gietzen <jgietzen@woot.com>
Date: Sep 12, 2013, 22:01 UTC
Message-ID: <D4C9DBA5F955904683C89B4A96DA7759EA78D60FDF@synapsemail>

Background: Windows, git version 1.8.3.msysgit.0 bare repo, 54k commits after migration from HG git filter-branch --prune-empty -- --all

I'm trying to clean up our repository after migrating it from HG.  I'm running the filter-branch command listed above in an effort to clean up all of garbage commits that HG required ("closing branch" commits and their ilk).

>From my past experience, "git filter-branch" is extremely quick when using simple filters, like env-filter, since it doesn't have to touch the working dir.  However, in our case each revision is taking 1-3 seconds; our entire repo will take 30 hours to clean up at this rate.  Normally, this wouldn't be a problem, except that we are getting "sh.exe couldn't start" errors after anywhere between the 5000th and 6000th rewritten commit.  Filter-branch doesn't have support for picking up where it left off, so we are entirely unable to clean up our repo.

All that being said, I have 3 questions:
  1.  Is there anything I can do to speed up the filter-branch command? (Alternatively, is there a way I can profile git-filter-branch.sh on msysgit?)
  2.  Any idea why sh.exe would fail?
  3.  Is there a way I can resume the filter-branch command when/if it fails?  (Alternatively, is there a way I can do the filter-branch in pieces and efficiently rebase... or something?)

I have already had to modify git-filter-branch.sh myself (to support the immense number of refs we are rewriting), so I'm comfortable with that.

Any help you can offer would be appreciated.
 
Thanks in advance,
John Gietzen