threads / patch / 44370

patchgit-svn: do not reuse caches memoized for a different architecture

Subject: [PATCH] git-svn: do not reuse caches memoized for a different architecture

## tl;dr

4 messages between Oct 25, 2016 and Oct 27, 2016. Diffs are folded; open one to read it.

replies: 3people: 3as markdown or json

Johannes Schindelin· Oct 25, 2016, 15:30 UTC · lore
From: Gavin Lambert <github@mirality.co.nz>

Reusing cached data speeds up git-svn by quite a fair bit. However, if the YAML module is unavailable, the caches are written to disk in an architecture-dependent manner. That leads to problems when upgrading, say, from 32-bit to 64-bit Git for Windows.

Let's just try to read those caches back if we detect the absence of the YAML module and the presence of the file, and delete the file if it could not be read back correctly.

Note that the only way to catch the error when the memoized cache could not be read back is to put the call inside an `eval { ... }` block because it would die otherwise; the `eval` block should also return `1` in case of success explicitly since the function reading back the cached data does not return an appropriate value to test for success.

This fixes https://github.com/git-for-windows/git/issues/233.
Signed-off-by: Gavin Lambert <github@mirality.co.nz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
---
Published-As: https://github.com/dscho/git/releases/tag/svn-multi-arch-v1
Fetch-It-Via: git fetch https://github.com/dscho/git svn-multi-arch-v1
We carried this for some time in Git for Windows.
 perl/Git/SVN.pm | 5 +++++
 1 file changed, 5 insertions(+)
Show changes to perl/Git/SVN.pm +5 −0
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 018beb8..025c894 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1658,6 +1658,11 @@ sub tie_for_persistent_memoization {
 	if ($memo_backend > 0) {
 		tie %$hash => 'Git::SVN::Memoize::YAML', "$path.yaml";
 	} else {
+		# first verify that any existing file can actually be loaded
+		# (it may have been saved by an incompatible version)
+		if (-e "$path.db") {
+			unlink "$path.db" unless eval { retrieve("$path.db"); 1 };
+		}
 		tie %$hash => 'Memoize::Storable', "$path.db", 'nstore';
 	}
 }

base-commit: 659889482ac63411daea38b2c3d127842ea04e4d

-- 
2.10.1.583.g721a9e0
Eric Wong· Oct 25, 2016, 21:23 UTC · re: Johannes Schindelin · lore

Re: [PATCH] git-svn: do not reuse caches memoized for a different architecture

Johannes Schindelin <johannes.schindelin@gmx.de> wrote:
Show 10 quoted lines
> +++ b/perl/Git/SVN.pm
> @@ -1658,6 +1658,11 @@ sub tie_for_persistent_memoization {
>  	if ($memo_backend > 0) {
>  		tie %$hash => 'Git::SVN::Memoize::YAML', "$path.yaml";
>  	} else {
> +		# first verify that any existing file can actually be loaded
> +		# (it may have been saved by an incompatible version)
> +		if (-e "$path.db") {
> +			unlink "$path.db" unless eval { retrieve("$path.db"); 1 };
> +		}

That retrieve() call is unlikely to work without "use Storable" to import it into the current package.

I also favor setting "$path.db" once to detect typos and avoid going over 80 columns. Additionally, having error-checking for unlink might be useful.

So perhaps squashing this on top:
Show changes to perl/Git/SVN.pm +8 −3
diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
index 025c894..b3c1460 100644
--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1660,10 +1660,15 @@ sub tie_for_persistent_memoization {
 	} else {
 		# first verify that any existing file can actually be loaded
 		# (it may have been saved by an incompatible version)
-		if (-e "$path.db") {
-			unlink "$path.db" unless eval { retrieve("$path.db"); 1 };
+		my $db = "$path.db";
+		if (-e $db) {
+			use Storable qw(retrieve);
+
+			if (!eval { retrieve($db); 1 }) {
+				unlink $db or die "unlink $db failed: $!";
+			}
 		}
-		tie %$hash => 'Memoize::Storable', "$path.db", 'nstore';
+		tie %$hash => 'Memoize::Storable', $db, 'nstore';
 	}
 }
 

Thoughts?  Thanks.
Junio C Hamano· Oct 27, 2016, 19:44 UTC · re: Eric Wong · lore

Re: [PATCH] git-svn: do not reuse caches memoized for a different architecture

Eric Wong <e@80x24.org> writes:
Show 46 quoted lines
> Johannes Schindelin <johannes.schindelin@gmx.de> wrote:
>> +++ b/perl/Git/SVN.pm
>> @@ -1658,6 +1658,11 @@ sub tie_for_persistent_memoization {
>>  	if ($memo_backend > 0) {
>>  		tie %$hash => 'Git::SVN::Memoize::YAML', "$path.yaml";
>>  	} else {
>> +		# first verify that any existing file can actually be loaded
>> +		# (it may have been saved by an incompatible version)
>> +		if (-e "$path.db") {
>> +			unlink "$path.db" unless eval { retrieve("$path.db"); 1 };
>> +		}
>
> That retrieve() call is unlikely to work without "use Storable"
> to import it into the current package.
>
> I also favor setting "$path.db" once to detect typos and avoid
> going over 80 columns.  Additionally, having error-checking for
> unlink might be useful.
>
> So perhaps squashing this on top:
>
> diff --git a/perl/Git/SVN.pm b/perl/Git/SVN.pm
> index 025c894..b3c1460 100644
> --- a/perl/Git/SVN.pm
> +++ b/perl/Git/SVN.pm
> @@ -1660,10 +1660,15 @@ sub tie_for_persistent_memoization {
>  	} else {
>  		# first verify that any existing file can actually be loaded
>  		# (it may have been saved by an incompatible version)
> -		if (-e "$path.db") {
> -			unlink "$path.db" unless eval { retrieve("$path.db"); 1 };
> +		my $db = "$path.db";
> +		if (-e $db) {
> +			use Storable qw(retrieve);
> +
> +			if (!eval { retrieve($db); 1 }) {
> +				unlink $db or die "unlink $db failed: $!";
> +			}
>  		}
> -		tie %$hash => 'Memoize::Storable', "$path.db", 'nstore';
> +		tie %$hash => 'Memoize::Storable', $db, 'nstore';
>  	}
>  }
>  
>
> Thoughts?  Thanks.

Just peeking from the sideline, but the your squash looks like an improvement to me.

Hopefully the final version after your interaction with Dscho can come to me via another "pull this now"?

Thanks.
Eric Wong· Oct 27, 2016, 20:51 UTC · re: Junio C Hamano · lore

Re: [PATCH] git-svn: do not reuse caches memoized for a different architecture

Junio C Hamano <gitster@pobox.com> wrote:
> Just peeking from the sideline, but the your squash looks like an
> improvement to me.
Thanks.
> Hopefully the final version after your interaction with Dscho can
> come to me via another "pull this now"?

Not sure if I'll be online the next few days, but I've preeptively pushed the patch + squash to my repo:

The following changes since commit 2cc2e70264e0fcba04f9ef791d144bbc8b501206:
  Eleventh batch for 2.11 (2016-10-26 13:28:47 -0700)
are available in the git repository at:
  git://bogomips.org/git-svn.git svn-cache
for you to fetch changes up to a2c761ce5b7a5fd8b505b036f3509a9e6617dee8:
  git-svn: do not reuse caches memoized for a different architecture (2016-10-27 20:17:36 +0000)
----------------------------------------------------------------
Gavin Lambert (1):
      git-svn: do not reuse caches memoized for a different architecture
 perl/Git/SVN.pm | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

← back to recent threads