threads / discuss / 37824

Re: Anomaly with the new code - Re: git-svn performance

Subject: Re: Anomaly with the new code - Re: git-svn performance

## tl;dr

14 messages between Oct 27, 2014 and Oct 30, 2014.

replies: 13people: 2as markdown or json

Hin-Tak Leung· Oct 27, 2014, 23:26 UTC · lore

------------------------------ On Mon, Oct 27, 2014 16:56 GMT Eric Wong wrote:

Show 10 quoted lines
>Eric Wong <normalperson@yhbt.net> wrote:
>> Which SVN version are you using?  I'm cloning (currently on r373xx)
>> https://svn.r-project.org/R using --stdlayout and
>> unable to see memory growth of the git-svn Perl process beyond 40M
>> (on a 32-bit system).
>
>git-svn hit 45M and took 11:44 to finish.   My ping times to
>svn.r-project.org is around 150ms (I'm running this from a server in
>Fremont, California).  I'll keep the repo around and periodically fetch
>to see how it runs.

I'll apply the 10 patches against 2.1.0 and see then. As I wrote in my last reply, my 3rd clone took about 8 hours to finish, and the max resident size is about 700MB (according to GNU "time").

AFAIK the hosting server is in northern Europe (Copahagen?), I think, so it is supposed to be faster for me fetching from UK.

Hin-Tak Leung· Oct 28, 2014, 05:40 UTC · re: Hin-Tak Leung · lore

differences between old clone and new Re: git-svn performance

To compare the old clone with the new, I did:
git branch -r | sort | xargs -n 1 git log --decorate=full -n 1

It turned out other than the empty vs 3 word commit messages about two years ago on trunk (which are inherited in all the newer branches), there are two other groups of differences.

One branch on the old clone has an extra merge from trunk ( and some extra trunk commits) listed in 'git log', while another branch has the exact opposite - on the old clone has one fewer merge.

I see the merge seem to be genuine - the subversion log often says so e.g. "ported from rXXX from trunk", but the extra/missing pattern isn't consistent.

So the histories are largely the same, except due to the extra merge, don't have the same sha1 sums.

Eric Wong· Oct 28, 2014, 07:41 UTC · re: Hin-Tak Leung · lore

Re: differences between old clone and new Re: git-svn performance

Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
Show 16 quoted lines
> To compare the old clone with the new, I did:
> 
> git branch -r | sort | xargs -n 1 git log --decorate=full -n 1
> 
> It turned out other than the empty vs 3 word commit messages
> about two years ago on trunk (which are inherited in all the newer
> branches), there are two other groups of differences.
> 
> One branch on the old clone has an extra merge from trunk (
> and some extra trunk commits) listed in 'git log', while
> another branch has the exact opposite - on the old clone
> has one fewer merge.
> 
> I see the merge seem to be genuine - the subversion log
> often says so e.g. "ported from rXXX from trunk", but
> the extra/missing pattern isn't consistent.

So both merges are correct, but we lose one, and gain one? I'll try to check more closely tomorrow. Can you point out the exact revisions in the R repo? Thanks.

> So the histories are largely the same, except due to the 
> extra merge, don't have the same sha1 sums.
Hin-Tak Leung· Oct 28, 2014, 23:59 UTC · re: Eric Wong · lore

Re: differences between old clone and new Re: git-svn performance

On Tue, 28/10/14, Eric Wong <normalperson@yhbt.net> wrote:
> So both merges
 are correct, but we lose one, and gain one?
 I'll try to check more closely tomorrow. 
 Can you point out
 the exact revisions in the
 R repo?  Thanks.
The missing merge on branch "R-2-14-branch" is:
commit 93af4d4cc3a5e0039944dd4e340d26995be8a252
Merge: 121990f 6ff1b87
Author: ripley <ripley@00db46b3-68df-0310-9c12-caf00c1e9a41>
Date:   Wed Feb 22 13:45:34 2012 +0000
    port r58453 from trunk
    git-svn-id: https://svn.r-project.org/R/branches/R-2-14-branch@58454 00db46b3-68df-0310-9c12-caf00c1e9a41

121990f is R-2-14-branch@58449, 6ff1b87 is trunk@58453, but the two branches are only identical up to and including R-2-14-branch@57129 - i.e. trunk@57130 appears in my old clone's "R-2-14-branch" git log but not in the new clone's.

The extra merge in the new clone is in branch "djm-parseRd":
commit 6d93330f7637eb4da81adaea58454c6b43da1c65
Merge: f503a9d 23deade
Author: murdoch <murdoch@00db46b3-68df-0310-9c12-caf00c1e9a41>
Date:   Thu Nov 13 14:24:17 2008 +0000
    Update from trunk to r46923
    
    git-svn-id: https://svn.r-project.org/R/branches/djm-parseRd@46925 00db46b3-68df-0310-9c12-caf00c1e9a41

You can look up f503a9d (djm-parseRd@46922) and 23deade (trunk@46923) yourself. The two branches' git log agree up to and including djm-parseRd@46659 .

Hmm, the new "djm-parseRd" branch actually have *two* extra merges, the earlier extra is:

commit 1e8174c797ba8471d604e89e4d614ad969b93b72
Merge: 55a1d9b 72744ab
Author: murdoch <murdoch@00db46b3-68df-0310-9c12-caf00c1e9a41>
Date:   Tue Nov 11 21:17:06 2008 +0000
    Merge trunk changes to r46902
    
    git-svn-id: https://svn.r-project.org/R/branches/djm-parseRd@46906 00db46b3-68df-0310-9c12-caf00c1e9a41
Eric Wong· Oct 30, 2014, 00:21 UTC · re: Hin-Tak Leung · lore

Re: differences between old clone and new Re: git-svn performance

Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
Show 20 quoted lines
> On Tue, 28/10/14, Eric Wong <normalperson@yhbt.net> wrote:
> 
> > So both merges
>  are correct, but we lose one, and gain one?
>  I'll try to check more closely tomorrow. 
>  Can you point out
>  the exact revisions in the
>  R repo?  Thanks.
> 
> 
> The missing merge on branch "R-2-14-branch" is:
> 
> commit 93af4d4cc3a5e0039944dd4e340d26995be8a252
> Merge: 121990f 6ff1b87
> Author: ripley <ripley@00db46b3-68df-0310-9c12-caf00c1e9a41>
> Date:   Wed Feb 22 13:45:34 2012 +0000
> 
>     port r58453 from trunk
> 
>     git-svn-id: https://svn.r-project.org/R/branches/R-2-14-branch@58454 00db46b3-68df-0310-9c12-caf00c1e9a41

I'm curious if you can tell me which version of git-svn you used to get that as a merge commit. git-svn mergeinfo handling has changed (hopefully improved) over the years, so some differences in history can be (unfortunately) expected, I think.

I cannot reproduce your original merge on Junio's current master. Using Junio's master[1] without any recent git-svn changes, a partial clone doing:

   git svn clone -s -r52000:58600 svn+ssh://127.0.0.1/path/to/my/R-mirror

...causes the merges in r58454 to be ignored as cherry-picks, too. I suspect it's correct for git-svn to ignore those as cherry-picks nowadays.

Here's a snippet of what I see from the above command:
----------------------------------8<-----------------------------------
r58452 = ebf3a1ca312ca7cc03dc2387d86491a0cdc95bad (refs/remotes/origin/trunk)
	M	src/library/base/man/Primitive.Rd
	M	src/main/names.c
	M	doc/NEWS.Rd
r58453 = 05b55eee9e6bed628873d34261e54c70f87a3736 (refs/remotes/origin/trunk)
	M	doc/NEWS.Rd
	M	src/library/base/man/Primitive.Rd
	M	src/main/names.c
W:svn cherry-pick ignored (/branches/R-2-12-branch:52939,54476,55265) - missing 492 commit(s) (eg df9d875de507ac51932c0ed980392e8262f98b31)
W:svn cherry-pick ignored (/branches/R-2-13-branch:55265,55432) - missing 231 commit(s) (eg cad052d416d9b8a9dfbfb2ae7bf85c39306c67bb)
W:svn cherry-pick ignored (/trunk:57183,57204-57205,57242,57259,57314,57316,57321,57370,57411,57428,57430,57432,57438,57440,57484,57489-57490,57579,57589,57604,57614-57618,57625,57679,57681,57687,57738,57741,57744-57745,57747,57752,57758,57761,57763,57765,57767,57769,57771,57790,57793,57803,57812,57814,57816,57826-57827,57836,57840-57841,57844,57846,57851,57853,57856,57861-57862,57867,57880,57884,57890,57893,57895,57900,57904,57908,57913,57920,57936,57939-57941,57950,57952,57959,57964,57970,57975,57977,57981,57987,58006,58008,58037,58039,58042,58047,58052,58056,58058,58066-58067,58082,58084,58089,58094,58098,58100,58107,58126,58129,58135,58142,58161,58178,58182,58187,58195,58204,58213,58217,58221,58225,58228,58232,58234,58239,58248,58253,58265,58269,58272,58274,58276,58278,58282,58284,58288,58294,58296,58305,58312,58314,58318,58324,58326,58328,58332,58334,58340,58346,58348,58353,58355,58357,58359,58361,58373,58378,58381,58386,58388,58392,58395,58397,58405,58412,58415,58429,58435,58437,58439,58453) - missing 716 commit(s) (eg e9ccca5db27696ed8faa4427ec4110ddf230d141)
r58454 = 96d6087a494bb7da6d90f02e8bd36833eaad2067 (refs/remotes/origin/R-2-14-branch)
	M	doc/manual/R-exts.texi
	M	doc/NEWS.Rd
	M	src/library/tools/R/check.R
r58455 = 742cbc791fa6760d5dfb4c4ea1e032d32e9e87c9 (refs/remotes/origin/trunk)
----------------------------------8<-----------------------------------
[1] - fbecd99 Update draft release notes to 2.2
Hin-Tak Leung· Oct 30, 2014, 00:55 UTC · re: Eric Wong · lore

Re: differences between old clone and new Re: git-svn performance

-------------------------------------------- On Thu, 30/10/14, Eric Wong <normalperson@yhbt.net> wrote:

 > The missing merge on branch
 "R-2-14-branch" is:
 > 
 > commit
 93af4d4cc3a5e0039944dd4e340d26995be8a252
 > Merge: 121990f 6ff1b87
 > Author: ripley <ripley@00db46b3-68df-0310-9c12-caf00c1e9a41>
 > Date:   Wed Feb 22 13:45:34
 2012 +0000
 > 
 > 
    port r58453 from trunk
 >
 
 >     git-svn-id: https://svn.r-project.org/R/branches/R-2-14-branch@58454
 00db46b3-68df-0310-9c12-caf00c1e9a41
 
> I'm curious if you can tell me which
 version of git-svn you used to get
 that as a
 merge commit.  git-svn mergeinfo handling has changed
 (hopefully improved) over the years, so some
 differences in history
 can be
 (unfortunately) expected, I think.

That's quite straight-forward, I think - except for the recent burst (I am essentially adapting the git 2.1.0 release shipped by the upcoming fedora 21 scheduled for christmas) I tend to update to the latest fedora release about a week or two after release; fedora 17 was shipped in May 2012 and only just enter Alpha in 22 Feb 2012. and I tracked R at least as frequently as weekly around then; So I would be using what ever version of git was shipping with fedora 16 around late Feb 2012.

On fedora's build farm, git-1.7.7.5 was bult in dec 2011 and git-1.7.7.6 was built on 2012-01-19 . Depending on how soon 1.7.7.6 filtered down to update, and when I update my git and also tracked R, (all three of these events probably happened around 22 Feb), I could be using either 1.7.7.5 or 1.7.7.6. I still have the system software update log around (the repo was cloned on a now-dead system, then moved over when it died), and presumably I can get git log to show me the fetch date (?), I might be able to tell whether it is 17.7.5 or 1.7.7.6 if you really want to know.

Eric Wong· Oct 30, 2014, 23:08 UTC · re: Hin-Tak Leung · lore

Re: differences between old clone and new Re: git-svn performance

Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
Show 16 quoted lines
> That's quite straight-forward, I think  - except for the recent burst (I am essentially
> adapting the git 2.1.0 release shipped by the upcoming fedora 21 scheduled for christmas)
> I tend to update to the latest fedora release about a week or two after release;
> fedora 17 was shipped in May 2012 and only just enter Alpha in 22 Feb 2012.
> and I tracked R at least as frequently as weekly around then;
> So I would be using what ever version of git was shipping with fedora 16 around late
> Feb 2012.
> 
> On fedora's build farm, git-1.7.7.5 was bult in dec 2011 and git-1.7.7.6 was built
> on 2012-01-19 . Depending on how soon
> 1.7.7.6 filtered down to update, and when I update my git and also tracked R,
> (all three of these events probably happened around 22 Feb), I could be
> using either 1.7.7.5 or 1.7.7.6. I still have the system software update log around
> (the repo was cloned on a now-dead system, then moved over when it died),
> and presumably I can get git log to show me the fetch date (?), I might
> be able to tell whether it is 17.7.5 or 1.7.7.6 if you really want to know.

I tried a full clone on 1.7.7.6 (no git-svn difference from 1.7.7.5). Even with that old git, I was able to reproduce the same merge behavior as current (Junio's) master as well as our recent patches.

So I believe r58454, r46925, and r46906 in the R repo are all handled correctly and no mergeinfo-handling regressions are introduced in the latest round of git-svn changes. Thanks.

Hin-Tak Leung· Oct 28, 2014, 23:33 UTC · re: Hin-Tak Leung · lore

Regression and failure to clone/fetch with new code Re: git-svn performance

Hi, I patched my system git with the recent git-svn improvements, and just use it for general use; so theses are the patches, against 2.1.0.

0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch 0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch 0004-git-svn-cache-only-mergeinfo-revisions.patch 0005-git-svn-remove-mergeinfo-rev-caching.patch 0006-git-svn.txt-advertise-pushurl-with-dcommit.patch 0007-git-svn-reload-RA-every-log-window-size.patch 0008-git-svn-remove-unnecessary-DESTROY-override.patch 0009-git-svn-save-a-little-memory-as-fetch-progresses.patch 0010-git-svn-disable-_rev_list-memoization.patch

trying to do this: git svn clone http://www.virtualbox.org/svn/vbox/trunk vbox

(there is no publicly visible branches, so it is just a straight-forward single-branch clone).
aborts with 
---------------
	M	src/VBox/Main/HostImpl.cpp
Incorrect parameters given: Could not convert '%ld' into a number at /usr/share/perl5/vendor_perl/Git/SVN.pm line 1711.
$ git svn fetch --all
Index mismatch: d6c75bc195b1daad647322e2cc025bd31265c6b9 != 3927d05f6ab037fcf2b4d964c9633efade037d1b
rereading a65b5fc0077c2fa80a344833b65ac19ff4ae88b6
	M	src/VBox/Main/HostImpl.cpp
Incorrect parameters given: Could not convert '%ld' into a number at /usr/share/perl5/vendor_perl/Git/SVN.pm line 1711.
----------------

I have never seen such behavior before, and seeing as the lines indicated are in a routine called "mergeinfo_changes", and recently added/changed by quite a few of the patches, I started reverting from the back in this order: #5, #4, #2, #1 and tried again between each revert. And it finally allows me to fetch again after reverting #1.

I don't see any %ld close by, but presumably this is enough information for somebody else to try. The platform is linux x86_64. (mostly fedora 20 but with a lot of additional changes like a newer gnome than shipped, etc so probably not really fc20)

Eric Wong· Oct 29, 2014, 19:23 UTC · re: Hin-Tak Leung · lore

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
Show 37 quoted lines
> Hi, I patched my system git with the recent git-svn improvements, and just use
> it for general use; so theses are the patches, against 2.1.0.
> 
> 0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch
> 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch
> 0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch
> 0004-git-svn-cache-only-mergeinfo-revisions.patch
> 0005-git-svn-remove-mergeinfo-rev-caching.patch
> 0006-git-svn.txt-advertise-pushurl-with-dcommit.patch
> 0007-git-svn-reload-RA-every-log-window-size.patch
> 0008-git-svn-remove-unnecessary-DESTROY-override.patch
> 0009-git-svn-save-a-little-memory-as-fetch-progresses.patch
> 0010-git-svn-disable-_rev_list-memoization.patch
> 
> trying to do this:
> git svn clone http://www.virtualbox.org/svn/vbox/trunk vbox
> 
> (there is no publicly visible branches, so it is just a straight-forward single-branch clone).
> 
> aborts with 
> 
> ---------------
> 	M	src/VBox/Main/HostImpl.cpp
> Incorrect parameters given: Could not convert '%ld' into a number at /usr/share/perl5/vendor_perl/Git/SVN.pm line 1711.
> 
> $ git svn fetch --all
> Index mismatch: d6c75bc195b1daad647322e2cc025bd31265c6b9 != 3927d05f6ab037fcf2b4d964c9633efade037d1b
> rereading a65b5fc0077c2fa80a344833b65ac19ff4ae88b6
> 	M	src/VBox/Main/HostImpl.cpp
> Incorrect parameters given: Could not convert '%ld' into a number at /usr/share/perl5/vendor_perl/Git/SVN.pm line 1711.
> ----------------
> 
> I have never seen such behavior before, and seeing as the lines indicated are in
> a routine called "mergeinfo_changes", and recently added/changed by
> quite a few of the patches, I started reverting from the back in this order: #5, #4, #2, #1 
> and tried again between each revert. And it finally allows me to fetch again after
> reverting #1.

Me neither, this is new bug to me. I cannot reproduce it, either. Which revision did you hit this on? I completed your vbox trunk clone without any problems on my side (Debian i386, SVN 1.6.17).

Can you try the following to dump out the parameters passed to mergeinfo_changes?

--- a/perl/Git/SVN.pm
+++ b/perl/Git/SVN.pm
@@ -1695,8 +1695,10 @@ sub parents_exclude {
 }
 
 # Compute what's new in svn:mergeinfo.
+use Data::Dumper;
 sub mergeinfo_changes {
 	my ($self, $old_path, $old_rev, $path, $rev, $mergeinfo_prop) = @_;
+	print STDERR Dumper(\@_);
 	my %minfo = map {split ":", $_ } split "\n", $mergeinfo_prop;
 	my $old_minfo = {};
 

Btw, I missed part of your other email, but no, I never maintained any
Chinese packages in Debian.

> I don't see any %ld close by, but presumably this is enough information for somebody else
> to try. The platform is linux x86_64. (mostly fedora 20 but with a lot of additional
> changes like a newer gnome than shipped, etc so probably not really fc20) 
>
Hin-Tak Leung· Oct 30, 2014, 00:06 UTC · re: Eric Wong · lore

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

Argh, sorry. I thought I included the info but I didn't.

git 2.1.0 + 10 patches aborts after trunk@28923 (i.e. failing to fetch 28924); if I revert the patches in that order (#5,#4,#2, #1) and retry in the middle, I have to revert all 4 to get 'git svn fetch' to continue on to 28924.

I tried --stdlayout (it seems that there were branches, but just merged and "deleted" according to the web code browsing interface) but it failed at the same revision.

I'll try the data dump and see what it gives me...

What do you think were missing in my e-mails? The differences of new clone against old is a missing merge in at R-2-14-branch@58454 , and two extra merges at djm-parseRd@46925 and djm-parseRd@46906 .

-------------------------------------------- On Wed, 29/10/14, Eric Wong <normalperson@yhbt.net> wrote:

 Subject: Re: Regression and failure to clone/fetch with new code Re: git-svn performance
 To: "Hin-Tak Leung" <htl10@users.sourceforge.net>
 Cc: stoklund@2pi.dk, fabian.schmied@gmail.com, git@vger.kernel.org, sam@vilain.net, stevenrwalter@gmail.com, waste.manager@gmx.de, amyrick@apple.com
 Date: Wednesday, 29 October, 2014, 20:23
 
 Hin-Tak Leung <htl10@users.sourceforge.net>
 wrote:
 > Hi, I patched my system git with
 the recent git-svn improvements, and just use
 > it for general use; so theses are the
 patches, against 2.1.0.
 > 
 >
 0001-git-svn-only-look-at-the-new-parts-of-svn-mergeinfo.patch
 >
 0002-git-svn-only-look-at-the-root-path-for-svn-mergeinfo.patch
 >
 0003-git-svn-reduce-check_cherry_pick-cache-overhead.patch
 >
 0004-git-svn-cache-only-mergeinfo-revisions.patch
 >
 0005-git-svn-remove-mergeinfo-rev-caching.patch
 >
 0006-git-svn.txt-advertise-pushurl-with-dcommit.patch
 >
 0007-git-svn-reload-RA-every-log-window-size.patch
 >
 0008-git-svn-remove-unnecessary-DESTROY-override.patch
 >
 0009-git-svn-save-a-little-memory-as-fetch-progresses.patch
 >
 0010-git-svn-disable-_rev_list-memoization.patch
 > 
 > trying to do
 this:
 > git svn clone http://www.virtualbox.org/svn/vbox/trunk
 vbox
 > 
 > (there
 is no publicly visible branches, so it is just a
 straight-forward single-branch clone).
 >
 
 > aborts with 
 > 
 > ---------------
 >
     M    src/VBox/Main/HostImpl.cpp
 > Incorrect parameters given: Could not
 convert '%ld' into a number at
 /usr/share/perl5/vendor_perl/Git/SVN.pm line 1711.
 > 
 > $ git svn fetch
 --all
 > Index mismatch:
 d6c75bc195b1daad647322e2cc025bd31265c6b9 !=
 3927d05f6ab037fcf2b4d964c9633efade037d1b
 > rereading
 a65b5fc0077c2fa80a344833b65ac19ff4ae88b6
 >     M   
 src/VBox/Main/HostImpl.cpp
 > Incorrect
 parameters given: Could not convert '%ld' into a
 number at /usr/share/perl5/vendor_perl/Git/SVN.pm line
 1711.
 > ----------------
 > 
 > I have never seen
 such behavior before, and seeing as the lines indicated are
 in
 > a routine called
 "mergeinfo_changes", and recently added/changed
 by
 > quite a few of the patches, I
 started reverting from the back in this order: #5, #4, #2,
 #1 
 > and tried again between each
 revert. And it finally allows me to fetch again after
 > reverting #1.
 
 Me neither, this is new bug to me.  I cannot
 reproduce it, either.  Which
 revision did
 you hit this on?  I completed your vbox trunk clone
 without
 any problems on my side (Debian
 i386, SVN 1.6.17).
 
 Can you
 try the following to dump out the parameters passed to
 mergeinfo_changes?
 
 --- a/perl/Git/SVN.pm
 +++
 b/perl/Git/SVN.pm
 @@ -1695,8 +1695,10 @@ sub
 parents_exclude {
  }
  
  # Compute what's new in svn:mergeinfo.
 +use Data::Dumper;
  sub
 mergeinfo_changes {
      my ($self,
 $old_path, $old_rev, $path, $rev, $mergeinfo_prop) = @_;
 +    print STDERR Dumper(\@_);
      my %minfo = map {split ":",
 $_ } split "\n", $mergeinfo_prop;
 
     my $old_minfo = {};
  
 
 Btw, I missed part of your
 other email, but no, I never maintained any
 Chinese packages in Debian.
 
 > I don't see any %ld close by, but
 presumably this is enough information for somebody else
 > to try. The platform is linux x86_64.
 (mostly fedora 20 but with a lot of additional
 > changes like a newer gnome than shipped,
 etc so probably not really fc20) 
 >
Eric Wong· Oct 30, 2014, 00:28 UTC · re: Hin-Tak Leung · lore

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
> Argh, sorry. I thought I included the info but I didn't.
Thanks. I'll try a different version of svn later.
> What do you think were missing in my e-mails?
I was skimming and missed the part about Debian packages :)
Hin-Tak Leung· Oct 30, 2014, 02:35 UTC · re: Eric Wong · lore

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

Here is the data dumper info . I tried the dumper code on the R repo as well, and saw that against the virtual box repo, there is one curious difference - $self->{last_rev} is a string rather than a number. I tried hacking around doing "$x += 0;" to coerce last_rev to a number at various places but didn't get very far. There seems to be some caching code in RA->get_dir so presumably that's why the same code run on one repo gives it as string while on another gives it a number. Hope you can figure where the coersion to string happened.

--------
$ git svn fetch --all
Index mismatch: d6c75bc195b1daad647322e2cc025bd31265c6b9 != 3927d05f6ab037fcf2b4d964c9633efade037d1b
rereading a65b5fc0077c2fa80a344833b65ac19ff4ae88b6
	M	src/VBox/Main/HostImpl.cpp
$VAR1 = [
          bless( {
                   'map_root' => '.git/svn/refs/remotes/origin/trunk/.rev_map',
                   '-use_svm_props' => undef,
                   'ra_uuid' => 'cfe28804-0f27-0410-a406-dd0f0b0b656f',
                   'pushurl' => undef,
                   '-follow_parent' => 1,
                   '-no_metadata' => undef,
                   '-rewrite_uuid' => undef,
                   'repo_id' => 'svn',
                   '-rewrite_root' => undef,
                   'index' => '.git/svn/refs/remotes/origin/trunk/index',
                   'dir' => '.git/svn/refs/remotes/origin/trunk',
                   'ref_id' => 'refs/remotes/origin/trunk',
                   'url' => 'http://www.virtualbox.org/svn/vbox',
                   'last_rev' => '28923',
                   'last_commit' => 'a65b5fc0077c2fa80a344833b65ac19ff4ae88b6',
                   '_path' => 'trunk',
                   'config' => '.git/svn/config',
                   'logged_rev_props' => {
                                           'log' => 'Main/Host: fix lock order issues in several methods
',
                                           'date' => '2010-04-30T09:08:17.252108Z',
                                           'author' => 'vboxsync'
                                         }
                 }, 'Git::SVN' ),
          'trunk',
          '28923',
          'trunk',
          28924,
          '/branches/VBox-3.0:58652'
        ];
Incorrect parameters given: Could not convert '%ld' into a number at /usr/share/perl5/vendor_perl/Git/SVN.pm line 1713.
------
--------------------------------------------
On Thu, 30/10/14, Eric Wong <normalperson@yhbt.net> wrote:
 
> I was skimming and missed the part about Debian
 packages :)

I only asked because you mentioned using Debian :-). The other 'Eric Wong' maintains/maintained a few chinese-related debian packages, from some years ago.

I know of two "Ken Sharp", one of ghostscript and another of wine, and two "David Turner", one of freetype and another of FSF's legal matters.

Eric Wong· Oct 30, 2014, 08:46 UTC · re: Hin-Tak Leung · lore

Re: Regression and failure to clone/fetch with new code Re: git-svn performance

Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
Show 8 quoted lines
> Here is the data dumper info . I tried the dumper code on the R repo
> as well, and saw that against the virtual box repo, there is one 
> curious difference - $self->{last_rev} is a string rather than a number.
> I tried hacking around doing "$x += 0;" to coerce last_rev
> to a number at various places but didn't get very far. There seems to be some caching
> code in RA->get_dir so presumably that's why the same code run
> on one repo gives it as string while on another gives it a number. Hope
> you can figure where the coersion to string happened.

Thanks, I'm not able to reproduce the issue, but can you try the following?

diff --git a/perl/Git/SVN/Ra.pm b/perl/Git/SVN/Ra.pm
index 75cdac9..82d6108 100644
--- a/perl/Git/SVN/Ra.pm
+++ b/perl/Git/SVN/Ra.pm
@@ -153,6 +153,7 @@ sub url {
 sub check_path {
 	my ($self, $path, $r) = @_;
 	my $cache = $self->{cache}->{check_path};
+	$r = int($r);
 	if ($r == $cache->{r} && exists $cache->{data}->{$path}) {
 		return $cache->{data}->{$path};
 	}
@@ -169,6 +170,7 @@ sub check_path {
 sub get_dir {
 	my ($self, $dir, $r) = @_;
 	my $cache = $self->{cache}->{get_dir};
+	$r = int($r);
 	if ($r == $cache->{r}) {
 		if (my $x = $cache->{data}->{$dir}) {
 			return wantarray ? @$x : $x->[0];
---
The above should apply to my current master which has some
minor cleanups (which I hope to send to Junio tomorrow).

The following changes since commit fbecd99861ea5795aeba46faf2ac7a8c1b70d485:

  Update draft release notes to 2.2 (2014-10-24 15:02:17 -0700)

are available in the git repository at:

  git://bogomips.org/git-svn.git master

for you to fetch changes up to da0bc948ac2e01652a150fd4a57cebad6143242c:

  git-svn: add space after "W:" prefix in warning (2014-10-30 08:31:28 +0000)

----------------------------------------------------------------
Eric Wong (11):
      git-svn: reduce check_cherry_pick cache overhead
      git-svn: cache only mergeinfo revisions
      git-svn: remove mergeinfo rev caching
      git-svn: reload RA every log-window-size
      git-svn: remove unnecessary DESTROY override
      git-svn: save a little memory as fetch progresses
      git-svn: disable _rev_list memoization
      Git.pm: add specified name to tempfile template
      git-svn: prepare SVN::Ra config pieces once
      git-svn: (cleanup) remove editor param passing
      git-svn: add space after "W:" prefix in warning

Jakob Stoklund Olesen (2):
      git-svn: only look at the new parts of svn:mergeinfo
      git-svn: only look at the root path for svn:mergeinfo

Sveinung Kvilhaugsvik (1):
      git-svn.txt: advertise pushurl with dcommit

 Documentation/git-svn.txt |   4 ++
 perl/Git.pm               |   5 +-
 perl/Git/SVN.pm           | 125 ++++++++++++++++++++++++++++------------------
 perl/Git/SVN/Ra.pm        |  90 ++++++++++++++++++---------------
 4 files changed, 134 insertions(+), 90 deletions(-)
Eric Wong· Oct 28, 2014, 07:45 UTC · re: Hin-Tak Leung · lore
Hin-Tak Leung <htl10@users.sourceforge.net> wrote:
Show 14 quoted lines
> >Eric Wong <normalperson@yhbt.net> wrote:
> >> Which SVN version are you using?  I'm cloning (currently on r373xx)
> >> https://svn.r-project.org/R using --stdlayout and
> >> unable to see memory growth of the git-svn Perl process beyond 40M
> >> (on a 32-bit system).
> >
> >git-svn hit 45M and took 11:44 to finish.   My ping times to
> >svn.r-project.org is around 150ms (I'm running this from a server in
> >Fremont, California).  I'll keep the repo around and periodically fetch
> >to see how it runs.
> 
> I'll apply the 10 patches against 2.1.0 and see then. As I wrote
> in my last reply, my 3rd clone took about 8 hours to finish,
> and the max resident size is about 700MB (according to GNU "time").

The "time" command is not a good measurement since it includes child process memory use (which may be file-backed mmap for git repack or "git cat-file --batch"). My measurements are just the RSS of the git-svn Perl process (from "ps aux" or VmRSS in /proc/$PID/status on Linux)

← back to recent threads