From: Eric Wong Date: Fri, 24 Oct 2014 23:34:11 GMT Subject: Re: Anomaly with the new code - Re: git-svn performance Message-ID: <20141024233411.GA18655@dcvr.yhbt.net> In-Reply-To: <1414134386.28852.YahooMailBasic@web172306.mail.ir2.yahoo.com> Hin-Tak Leung wrote: > I keep tabs of a particular svn repository over many years > and run "git svn fetch --all" every few days. So that's the old clone. > Since this discussion started, I made a new one with git 2.1.0 patched > with the first two patches below, a couple of weeks ago. And I ran > 'git svn fetch --all' on both every few days since. > > I have added a few more patches, so the whole list is the 6 > below against 2.1.0. The latest fetch is really strange - the fetch against > the new clone took almost twice as long and uses almost twice > as much memory, vs against the old. 17 min, 800 MB vs 10 min 400MB. > Details below. Maybe this is a performance issue about how the clones > were made? Memory usage seems to grow with the amount of revisions fetched, see below. And higher memory means slower fork() on Linux systems. > 0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch > 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch > 0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch > 0004-git-svn-cache-only-mergeinfo-revisions.patch > 0006-git-svn-clear-global-SVN-pool-between-get_log-invoca.patch 0006 is insufficient and incompatible with older SVN. I pushed "git-svn: reload RA every log-window-size" (commit dfa72fdb96befbd790f623bb2909a347176753c2) instead which saves much more memory: http://mid.gmane.org/20141024225352.GB31716@dcvr.yhbt.net But there still seems to be some slow growth with many revisions which is not mergeinfo-related. > 0007-git-svn-remove-mergeinfo-rev-caching.patch I think it is also safe to remove the _rev_list memoization since it uses a lot of memory. The remaining caches should be tiny (but useful, I think).