{"thread":{"id":"8538","subject":"[PATCH] git-svn: cache max revision in rev_db databases","startedAt":"2007-06-10T09:04:54Z","lastAt":"2007-06-11T11:43:54Z","messageCount":3,"participants":["Sam Vilain","Eric Wong"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"44568","messageId":"20070610091304.C3E0413A4F8@magnus.utsl.gen.nz","threadId":"8538","inReplyTo":null,"subject":"[PATCH] git-svn: cache max revision in rev_db databases","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2007-06-10T09:04:54Z","receivedAt":"2007-06-10T09:04:54Z","isPatch":true,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"Cache the maximum revision for each rev_db URL rather than looking it\nup each time.  This saves a lot of time when rebuilding indexes on a\nfreshly cloned repository.\n\nSigned-off-by: Sam Vilain <sam@vilain.net>\n---\n git-svn.perl |    4 ++++\n 1 files changed, 4 insertions(+), 0 deletions(-)\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex 610563c..c9758a0 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -804,6 +804,7 @@ sub working_head_info {\n \tmy ($head, $refs) = @_;\n \tmy ($fh, $ctx) = command_output_pipe('log', $head);\n \tmy $hash;\n+\tmy %max;\n \twhile (<$fh>) {\n \t\tif ( m{^commit ($::sha1)$} ) {\n \t\t\t$hash = $1;\n@@ -812,11 +813,14 @@ sub working_head_info {\n \t\tnext unless s{^\\s+(git-svn-id:)}{$1};\n \t\tmy ($url, $rev, $uuid) = extract_metadata($_);\n \t\tif (defined $url && defined $rev) {\n+\t\t\tnext if $max{$url} and $max{$url} < $rev;\n \t\t\tif (my $gs = Git::SVN->find_by_url($url)) {\n \t\t\t\tmy $c = $gs->rev_db_get($rev);\n \t\t\t\tif ($c && $c eq $hash) {\n \t\t\t\t\tclose $fh; # break the pipe\n \t\t\t\t\treturn ($url, $rev, $uuid, $gs);\n+\t\t\t\t} else {\n+\t\t\t\t\t$max{$url} ||= $gs->rev_db_max;\n \t\t\t\t}\n \t\t\t}\n \t\t}\n-- \n1.5.0.4.210.gf8a7c-dirty\n"},{"id":"44649","messageId":"20070610214753.GC12222@muzzle","threadId":"8538","inReplyTo":"20070610091304.C3E0413A4F8@magnus.utsl.gen.nz","subject":"Re: [PATCH] git-svn: cache max revision in rev_db databases","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2007-06-10T21:47:53Z","receivedAt":"2007-06-10T21:47:53Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Sam Vilain <sam@vilain.net> wrote:\n> Cache the maximum revision for each rev_db URL rather than looking it\n> up each time.  This saves a lot of time when rebuilding indexes on a\n> freshly cloned repository.\n\nIt looks correct, but I'm not sure how often we can even hit this\noptimization to make it worth it.\n\nAny live repositories and benchmarks you've run this on?\n(without the rev-list => log patch applied, which would be a big\nperformance improvement if all log messages were under 16k).\n\n> Signed-off-by: Sam Vilain <sam@vilain.net>\n> ---\n>  git-svn.perl |    4 ++++\n>  1 files changed, 4 insertions(+), 0 deletions(-)\n> \n> diff --git a/git-svn.perl b/git-svn.perl\n> index 610563c..c9758a0 100755\n> --- a/git-svn.perl\n> +++ b/git-svn.perl\n> @@ -804,6 +804,7 @@ sub working_head_info {\n>  \tmy ($head, $refs) = @_;\n>  \tmy ($fh, $ctx) = command_output_pipe('log', $head);\n>  \tmy $hash;\n> +\tmy %max;\n>  \twhile (<$fh>) {\n>  \t\tif ( m{^commit ($::sha1)$} ) {\n>  \t\t\t$hash = $1;\n> @@ -812,11 +813,14 @@ sub working_head_info {\n>  \t\tnext unless s{^\\s+(git-svn-id:)}{$1};\n>  \t\tmy ($url, $rev, $uuid) = extract_metadata($_);\n>  \t\tif (defined $url && defined $rev) {\n> +\t\t\tnext if $max{$url} and $max{$url} < $rev;\n>  \t\t\tif (my $gs = Git::SVN->find_by_url($url)) {\n>  \t\t\t\tmy $c = $gs->rev_db_get($rev);\n>  \t\t\t\tif ($c && $c eq $hash) {\n>  \t\t\t\t\tclose $fh; # break the pipe\n>  \t\t\t\t\treturn ($url, $rev, $uuid, $gs);\n> +\t\t\t\t} else {\n> +\t\t\t\t\t$max{$url} ||= $gs->rev_db_max;\n>  \t\t\t\t}\n>  \t\t\t}\n>  \t\t}\n\n-- \nEric Wong\n"},{"id":"44719","messageId":"466D357A.8070605@vilain.net","threadId":"8538","inReplyTo":"20070610214753.GC12222@muzzle","subject":"Re: [PATCH] git-svn: cache max revision in rev_db databases","fromName":"Sam Vilain","fromEmail":"sam@vilain.net","sentAt":"2007-06-11T11:43:54Z","receivedAt":"2007-06-11T11:43:54Z","isPatch":true,"sender":{"key":"sam@vilain.net","avatar":"https://gravatar.com/avatar/8fc840ca854dbf6f7065b4335e3b934951c1dca3b11db688e95e471901f8f4a8?d=mp&s=160"},"body":"Eric Wong wrote:\n> Sam Vilain <sam@vilain.net> wrote:\n>   \n>> Cache the maximum revision for each rev_db URL rather than looking it\n>> up each time.  This saves a lot of time when rebuilding indexes on a\n>> freshly cloned repository.\n>>     \n>\n> It looks correct, but I'm not sure how often we can even hit this\n> optimization to make it worth it.\n>\n> Any live repositories and benchmarks you've run this on?\n> (without the rev-list => log patch applied, which would be a big\n> performance improvement if all log messages were under 16k).\n>   \nIt may not make a huge difference with the current code, but it might\nwith a different rev_db back-end with higher connection cost, and it\nseemed like an obvious and innocuous enough memoization.\n\nSam\n"}]}