{"thread":{"id":"37817","subject":"[PATCH] git-svn: disable _rev_list memoization","startedAt":"2014-10-27T01:40:33Z","lastAt":"2014-10-27T01:40:33Z","messageCount":1,"participants":["Eric Wong"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"251084","messageId":"20141027014033.GA4189@dcvr.yhbt.net","threadId":"37817","inReplyTo":null,"subject":"[PATCH] git-svn: disable _rev_list memoization","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2014-10-27T01:40:33Z","receivedAt":"2014-10-27T01:40:33Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"This memoization appears unneeded as the check_cherry_pick2 cache is\nin front of it does enough.\n\nWith this change applied, importing from local svn+ssh and http copies\nof the R repo[1] takes only 2:00 (2 hours) on my system and the git-svn\nprocess never uses more than 60MB RSS on my x86-64 GNU/Linux system[2].\nThis 60M measurement is only for the git-svn Perl process itself and\ndoes not include memory used by git subprocesses accessing large packs\n(subprocess memory usage _is_ measured by my time(1) tool).\n\nBefore this change, an import took longer (2:20) on svn+ssh:// but\ngit-svn used around 240MB during the imports.  Worse yet, git-svn\nballooned to over 400M when writing out the cache to the filesystem.\n\nI also tried removing memoization for `has_no_changes', too, but a\nlocal copy of the R repository(*) was not close to finishing within\n10 hours on my system.\n\n[1] http://svn.r-project.org/R\n[2] file:// repos causes libsvn to use more memory internally\n\nSigned-off-by: Eric Wong <normalperson@yhbt.net>\nCc: Hin-Tak Leung <htl10@users.sourceforge.net>\n---\n  This patch is in my \"less-memo\" branch of bogomips.org/git-svn.git\n  Info here:\n\n  (not intended as a pull request for Junio, yet)\n\n  The following changes since commit fbecd99861ea5795aeba46faf2ac7a8c1b70d485:\n\n    Update draft release notes to 2.2 (2014-10-24 15:02:17 -0700)\n\n  are available in the git repository at:\n\n    git://bogomips.org/git-svn.git less-memo\n\n  for you to fetch changes up to 7676aff70973e617c3f58a8633db6d0e3ee99e45:\n\n    git-svn: disable _rev_list memoization (2014-10-27 01:39:39 +0000)\n\n  ----------------------------------------------------------------\n  Eric Wong (7):\n        git-svn: reduce check_cherry_pick cache overhead\n        git-svn: cache only mergeinfo revisions\n        git-svn: remove mergeinfo rev caching\n        git-svn: reload RA every log-window-size\n        git-svn: remove unnecessary DESTROY override\n        git-svn: save a little memory as fetch progresses\n        git-svn: disable _rev_list memoization\n\n  Jakob Stoklund Olesen (2):\n        git-svn: only look at the new parts of svn:mergeinfo\n        git-svn: only look at the root path for svn:mergeinfo\n\n  Sveinung Kvilhaugsvik (1):\n        git-svn.txt: advertise pushurl with dcommit\n\n   Documentation/git-svn.txt |   4 ++\n   perl/Git/SVN.pm           | 121 +++++++++++++++++++++++++++++-----------------\n   perl/Git/SVN/Ra.pm        |  27 ++++++-----\n   3 files changed, 94 insertions(+), 58 deletions(-)\n\n perl/Git/SVN.pm | 10 ----------\n 1 file changed, 10 deletions(-)\n\ndiff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm\nindex 4364506..5f9d469 100644\n--- a/perl/Git/SVN.pm\n+++ b/perl/Git/SVN.pm\n@@ -1600,7 +1600,6 @@ sub tie_for_persistent_memoization {\n \t\tmy %lookup_svn_merge_cache;\n \t\tmy %check_cherry_pick2_cache;\n \t\tmy %has_no_changes_cache;\n-\t\tmy %_rev_list_cache;\n \n \t\ttie_for_persistent_memoization(\\%lookup_svn_merge_cache,\n \t\t    \"$cache_path/lookup_svn_merge\");\n@@ -1622,14 +1621,6 @@ sub tie_for_persistent_memoization {\n \t\t\tSCALAR_CACHE => ['HASH' => \\%has_no_changes_cache],\n \t\t\tLIST_CACHE => 'FAULT',\n \t\t;\n-\n-\t\ttie_for_persistent_memoization(\\%_rev_list_cache,\n-\t\t    \"$cache_path/_rev_list\");\n-\t\tmemoize '_rev_list',\n-\t\t\tSCALAR_CACHE => 'FAULT',\n-\t\t\tLIST_CACHE => ['HASH' => \\%_rev_list_cache],\n-\t\t;\n-\n \t}\n \n \tsub unmemoize_svn_mergeinfo_functions {\n@@ -1639,7 +1630,6 @@ sub tie_for_persistent_memoization {\n \t\tMemoize::unmemoize 'lookup_svn_merge';\n \t\tMemoize::unmemoize 'check_cherry_pick2';\n \t\tMemoize::unmemoize 'has_no_changes';\n-\t\tMemoize::unmemoize '_rev_list';\n \t}\n \n \tsub clear_memoized_mergeinfo_caches {\n-- \nEW\n"}]}