threads / patch / 8538

patchgit-svn: cache max revision in rev_db databases

Subject: [PATCH] git-svn: cache max revision in rev_db databases

## tl;dr

3 messages between Jun 10, 2007 and Jun 11, 2007. Diffs are folded; open one to read it.

replies: 2people: 2as markdown or json

Sam Vilain· Jun 10, 2007, 09:04 UTC · lore

Cache the maximum revision for each rev_db URL rather than looking it up each time. This saves a lot of time when rebuilding indexes on a freshly cloned repository.

Signed-off-by: Sam Vilain <sam@vilain.net>
---
 git-svn.perl |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
Show changes to git-svn.perl +4 −0
diff --git a/git-svn.perl b/git-svn.perl
index 610563c..c9758a0 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -804,6 +804,7 @@ sub working_head_info {
 	my ($head, $refs) = @_;
 	my ($fh, $ctx) = command_output_pipe('log', $head);
 	my $hash;
+	my %max;
 	while (<$fh>) {
 		if ( m{^commit ($::sha1)$} ) {
 			$hash = $1;
@@ -812,11 +813,14 @@ sub working_head_info {
 		next unless s{^\s+(git-svn-id:)}{$1};
 		my ($url, $rev, $uuid) = extract_metadata($_);
 		if (defined $url && defined $rev) {
+			next if $max{$url} and $max{$url} < $rev;
 			if (my $gs = Git::SVN->find_by_url($url)) {
 				my $c = $gs->rev_db_get($rev);
 				if ($c && $c eq $hash) {
 					close $fh; # break the pipe
 					return ($url, $rev, $uuid, $gs);
+				} else {
+					$max{$url} ||= $gs->rev_db_max;
 				}
 			}
 		}
-- 
1.5.0.4.210.gf8a7c-dirty
Eric Wong· Jun 10, 2007, 21:47 UTC · re: Sam Vilain · lore

Re: [PATCH] git-svn: cache max revision in rev_db databases

Sam Vilain <sam@vilain.net> wrote:
> Cache the maximum revision for each rev_db URL rather than looking it
> up each time.  This saves a lot of time when rebuilding indexes on a
> freshly cloned repository.

It looks correct, but I'm not sure how often we can even hit this optimization to make it worth it.

Any live repositories and benchmarks you've run this on? (without the rev-list => log patch applied, which would be a big performance improvement if all log messages were under 16k).

Show 32 quoted lines
> Signed-off-by: Sam Vilain <sam@vilain.net>
> ---
>  git-svn.perl |    4 ++++
>  1 files changed, 4 insertions(+), 0 deletions(-)
> 
> diff --git a/git-svn.perl b/git-svn.perl
> index 610563c..c9758a0 100755
> --- a/git-svn.perl
> +++ b/git-svn.perl
> @@ -804,6 +804,7 @@ sub working_head_info {
>  	my ($head, $refs) = @_;
>  	my ($fh, $ctx) = command_output_pipe('log', $head);
>  	my $hash;
> +	my %max;
>  	while (<$fh>) {
>  		if ( m{^commit ($::sha1)$} ) {
>  			$hash = $1;
> @@ -812,11 +813,14 @@ sub working_head_info {
>  		next unless s{^\s+(git-svn-id:)}{$1};
>  		my ($url, $rev, $uuid) = extract_metadata($_);
>  		if (defined $url && defined $rev) {
> +			next if $max{$url} and $max{$url} < $rev;
>  			if (my $gs = Git::SVN->find_by_url($url)) {
>  				my $c = $gs->rev_db_get($rev);
>  				if ($c && $c eq $hash) {
>  					close $fh; # break the pipe
>  					return ($url, $rev, $uuid, $gs);
> +				} else {
> +					$max{$url} ||= $gs->rev_db_max;
>  				}
>  			}
>  		}
-- 
Eric Wong
Sam Vilain· Jun 11, 2007, 11:43 UTC · re: Eric Wong · lore

Re: [PATCH] git-svn: cache max revision in rev_db databases

Eric Wong wrote:
Show 14 quoted lines
> Sam Vilain <sam@vilain.net> wrote:
>   
>> Cache the maximum revision for each rev_db URL rather than looking it
>> up each time.  This saves a lot of time when rebuilding indexes on a
>> freshly cloned repository.
>>     
>
> It looks correct, but I'm not sure how often we can even hit this
> optimization to make it worth it.
>
> Any live repositories and benchmarks you've run this on?
> (without the rev-list => log patch applied, which would be a big
> performance improvement if all log messages were under 16k).
>   

It may not make a huge difference with the current code, but it might with a different rev_db back-end with higher connection cost, and it seemed like an obvious and innocuous enough memoization.

Sam

← back to recent threads