git/list[1] front-page[2] threads[3] people[4] search[5] about
 

git-svn: persistent memoization

From
Andrew Myrick <amyrick@apple.com>
Date
Jan 19, 2010, 23:37 UTC
Message-ID
<26542E21-CAF5-4ACD-B9DC-1981AE41F50C@apple.com>
git-svn uses the Memoize perl module to cache the return values of a few functions.  This speeds up svn:mergeinfo processing considerably, but as currently implemented, this memoization table must be reconstructed on every run of git-svn.  This isn't a problem on small projects, but it introduces a delay of several minutes when fetching from large repositories with a lot of merge info.
The Memoize module has support for storing its cache persistently (more below, and see http://perldoc.perl.org/Memoize.html for details), and I would love to take advantage of this support in git-svn.  Unfortunately, it's not entirely obvious to me how to get this working.  Here are the questions I have so far, moving from high level design questions to implementation details:
1) Is there any reason not to store these caches on disk?
2) Are there situations where the caches would need to be invalidated?  Perhaps when git-svn rebuilds its metadata?
3) Where should I put these caches?  I was thinking something like ".git/svn/caches/<cachefile>.db" would be appropriate.
4) What's the correct way to reference the path to those caches?  I tried using "$ENV{GIT_DIR}/svn/caches/has_no_changes.db", but the memoize calls are in a BEGIN block, and it seems that $ENV{GIT_DIR} hadn't been initialized at that point.  This has taken me way past my limited knowledge of Perl.
5) What backend should I pick to store the cache?  Memoize supports storing to any tied hash that supports TIEHASH, FETCH, STORE, and EXISTS, like DB_File.  It also supports storing to SDBM_File and NDBM_File through glue modules (they lack EXISTS support), and Storable, which isn't a tied hash at all, but just a serializer for hashes.  I'm leaning towards using Storable since it seems simple and would fit the workload well, but I would appreciate any insight from someone with more domain knowledge here.

Regards, Andrew

Message 1 of 1 in “git-svn: persistent memoization”
  1. Andrew MyrickJan 19, 2010

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.