Re: git-svn performance

From: Eric Wong <normalperson@yhbt.net>
Date: Oct 19, 2014, 00:32 UTC
Message-ID: <20141019003256.GA18532@dcvr.yhbt.net>
In-Reply-To: <CABBCAiv0WXNzo7W9PB_o_enLjtUO_rNRb4UBEqDPeSkBj1k-Ww@mail.gmail.com>

Fabian Schmied <fabian.schmied@gmail.com> wrote:

Show 23 quoted lines

> Hi,
> 
> I'm currently migrating an SVN repository to Git using git-svn (Git
> for Windows 1.8.3-preview20130601), and I'm experiencing severe
> performance problems with "git svn fetch". Commits to the SVN "trunk"
> are fetched very fast (a few seconds or so per SVN revision), but
> commits to some branches ("hotfix" branches) are currently taking
> about 9 minutes per revision. I fear that the time per these commits
> is increasing and that indeed the migration might not be finishable at
> all.
> 
> For the commits that take such a long time, git-svn always outputs
> lots of warnings about ignored SVN cherry-picks, and it tells me it
> can't find a revmap for the path being imported. (See [1].)
> 
> AFAICS, the offending commits take place on some branches that include
> a lot of manually merged ("SVN cherry-picked") revisions. Git-svn
> seems to be checking something (though I don't know what) that makes
> importing these revisions really slow. And it repeats this for every
> revision on these branches with increasing work to do.
> 
> Is there anything I can do to speed this up? (I already tried
> increasing the --log-window-size to 500, didn't have any effect.)

Can you take a look at the following two "mergeinfo-speedups" in my repo? (git://bogomips.org/git-svn)

Jakob Stoklund Olesen (2):
      git-svn: only look at the new parts of svn:mergeinfo
      git-svn: only look at the root path for svn:mergeinfo

Also downloadable here:

http://bogomips.org/git-svn.git/patch?id=9b258e721b30785357535 http://bogomips.org/git-svn.git/patch?id=73409a2145e93b436d74a

Hin-Tak (Cc-ed) reported good improvements with them, but also a large memory increase:

http://mid.gmane.org/1412706046.90413.YahooMailBasic@web172303.mail.ir2.yahoo.com

Jakob (or anybody else): I suppose we could tie the new cached_mergeinfo* caches to disk-backed storage to avoid the memory bloat.