threads / discuss / 21574

git-svn problem with v1.6.5

Subject: git-svn problem with v1.6.5

## tl;dr

17 messages between Nov 10, 2009 and Nov 12, 2009.

replies: 16people: 4as markdown or json

Pascal Obry· Nov 10, 2009, 22:23 UTC · lore
Hello,

A git-svn mirror is created using Git v1.6.4. When I copy this mirror into another machine which is using v1.6.5 I get the following error:

$ git svn rebase fatal: Invalid revision range d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk rev-list --pretty=raw --no-color --reverse d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk --: command returned error: 128

Using v1.6.4 instead I have no problem. Any idea to track this problem down?
Thanks.
Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B
Avery Pennarun· Nov 10, 2009, 22:28 UTC · re: Pascal Obry · lore

Re: git-svn problem with v1.6.5

On Tue, Nov 10, 2009 at 5:23 PM, Pascal Obry <pascal@obry.net> wrote:
Show 6 quoted lines
> $ git svn rebase
> fatal: Invalid revision range
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk
> rev-list --pretty=raw --no-color --reverse
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk --: command
> returned error: 128

Is d2cf08bb67e4b7da33a250127aab784f1f2f58d3 a valid revision? (git log d2cf08bb67e4b7da33a250127aab784f1f2f58d3). Is refs/remotes/svn/trunk valid? (git log refs/remotes/svn/trunk). It seems like a strange problem.

> Using v1.6.4 instead I have no problem. Any idea to track this problem down?

You could try using git bisect to figure out which exact commit to git.git created the problem.

Avery
Pascal Obry· Nov 11, 2009, 10:28 UTC · re: Avery Pennarun · lore

Re: git-svn problem with v1.6.5

Avery,
 > Is d2cf08bb67e4b7da33a250127aab784f1f2f58d3 a valid revision?  (git
 > log d2cf08bb67e4b7da33a250127aab784f1f2f58d3).
No.
 > Is refs/remotes/svn/trunk valid?  (git log refs/remotes/svn/trunk).
Yes.
 > It seems like a strange problem.
Agreed!
> You could try using git bisect to figure out which exact commit to
> git.git created the problem.
I wanted to avoid that :) Anyway, here is the culprit:
commit 6f5748e14cc5bb0a836b649fb8e2d6a5eb166f1d
Author: Adam Brewster <adambrewster@gmail.com>
Date:   Tue Aug 11 23:14:27 2009 -0400
     svn: allow branches outside of refs/remotes
     It may be convenient for some users to store svn remote tracking
     branches outside of the refs/remotes/ heirarchy.
     To accomplish this feat, this patch includes the entire path to
     the ref in $r->{'refname'} in &read_all_remotes and tries to change
     references to this entry so the new value makes sense.
     [ew: fixed backwards compatibility, long lines]
     Signed-off-by: Adam Brewster <adambrewster@gmail.com>
     Signed-off-by: Eric Wong <normalperson@yhbt.net>
  git-svn.perl                     |   83 
++++++++++++++++++++++----------------
  t/lib-git-svn.sh                 |    2 +-
  t/t9104-git-svn-follow-parent.sh |   10 ++--
  t/t9107-git-svn-migrate.sh       |   14 +++----
  t/t9143-git-svn-gc.sh            |   10 ++--
  t/t9144-git-svn-old-rev_map.sh   |   31 ++++++++++++++
  6 files changed, 96 insertions(+), 54 deletions(-)

But frankly I'm no expert on Perl... I fear that I won't be able to debug that!

Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B
Avery Pennarun· Nov 11, 2009, 19:41 UTC · re: Pascal Obry · lore

Re: git-svn problem with v1.6.5

On Wed, Nov 11, 2009 at 5:28 AM, Pascal Obry <pascal@obry.net> wrote:
Show 5 quoted lines
>> Avery wrote:
>> Is d2cf08bb67e4b7da33a250127aab784f1f2f58d3 a valid revision?  (git
>> log d2cf08bb67e4b7da33a250127aab784f1f2f58d3).
>
> No.

Well, that's bad news. Does "git log --all --parents | grep d2cf08bb67e4b7da33a250127aab784f1f2f58d3" reveal any places that refer to it?

It sounds a bit like your git-svn thinks something maps on to that commit id, but a previous 'git gc' or something has thrown it away. However, that doesn't explain why earlier git versions don't have this problem.

Show 10 quoted lines
>> You could try using git bisect to figure out which exact commit to
>> git.git created the problem.
>
> I wanted to avoid that :) Anyway, here is the culprit:
>
> commit 6f5748e14cc5bb0a836b649fb8e2d6a5eb166f1d
> Author: Adam Brewster <adambrewster@gmail.com>
> Date:   Tue Aug 11 23:14:27 2009 -0400
>
>   svn: allow branches outside of refs/remotes
[...]
>
> But frankly I'm no expert on Perl... I fear that I won't be able to debug
> that!

If you retrieve the latest version of git and then git revert the above commit, does that fix the problem, at least?

Is it possible you have some weird branches outside of the refs/remotes tree (either in .git itself, or in .git/refs/*) that you forgot about, and which the new version of git-svn is finding somehow?

Avery
Pascal Obry· Nov 11, 2009, 19:58 UTC · re: Avery Pennarun · lore

Re: git-svn problem with v1.6.5

Avery,
> Well, that's bad news.  Does "git log --all --parents | grep
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3" reveal any places that refer
> to it?
No.
> It sounds a bit like your git-svn thinks something maps on to that
> commit id, but a previous 'git gc' or something has thrown it away.
> However, that doesn't explain why earlier git versions don't have this
> problem.
Maybe, but that would be a quite annoying bug!
> If you retrieve the latest version of git and then git revert the
> above commit, does that fix the problem, at least?

I cannot, this does not revert cleanly and I don't know how to properly resolve this conflict. I'm no Perl expert!

But reverting using the version just before fixes this problem.
> Is it possible you have some weird branches outside of the
> refs/remotes tree (either in .git itself, or in .git/refs/*) that you
> forgot about, and which the new version of git-svn is finding somehow?
I do not see something under .git/refs/* (only empty directories).
The project has been imported using something like this:
  $ git svn clone --prefix=svn/ svn+ssh://server/path \
	--revision=580:HEAD \
	--trunk=trunk/project \
	--tags=tags/project \
	--branches=branches/project \
	--branches="branches/global/*/project" project

Nothing fancy. The git-svn mirrors is updated every night using Git 1.6.4 without problem.

Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B
Eric Wong· Nov 11, 2009, 20:22 UTC · re: Pascal Obry · lore

Re: git-svn problem with v1.6.5

Pascal Obry <pascal@obry.net> wrote:
Show 7 quoted lines
> Avery,
>
>> Well, that's bad news.  Does "git log --all --parents | grep
>> d2cf08bb67e4b7da33a250127aab784f1f2f58d3" reveal any places that refer
>> to it?
>
> No.
Weird...
Show 6 quoted lines
>> It sounds a bit like your git-svn thinks something maps on to that
>> commit id, but a previous 'git gc' or something has thrown it away.
>> However, that doesn't explain why earlier git versions don't have this
>> problem.
>
> Maybe, but that would be a quite annoying bug!
Show 7 quoted lines
>> If you retrieve the latest version of git and then git revert the
>> above commit, does that fix the problem, at least?
>
> I cannot, this does not revert cleanly and I don't know how to properly  
> resolve this conflict. I'm no Perl expert!
>
> But reverting using the version just before fixes this problem.
That should rule out filesystem corruption...
Show 14 quoted lines
>> Is it possible you have some weird branches outside of the
>> refs/remotes tree (either in .git itself, or in .git/refs/*) that you
>> forgot about, and which the new version of git-svn is finding somehow?
>
> I do not see something under .git/refs/* (only empty directories).
>
> The project has been imported using something like this:
>
>  $ git svn clone --prefix=svn/ svn+ssh://server/path \
> 	--revision=580:HEAD \
> 	--trunk=trunk/project \
> 	--tags=tags/project \
> 	--branches=branches/project \
> 	--branches="branches/global/*/project" project

It looks like you didn't use any of the weird[1] options that make it unsafe to clobber the rev_maps. Can you try removing all the .rev_map.* files in $GIT_DIR/svn and running "git svn fetch" to rebuild them?

[1] --no-metadata, --use-svm-props
-- 
Eric Wong
Eric Wong· Nov 11, 2009, 20:34 UTC · re: Eric Wong · lore

Re: git-svn problem with v1.6.5

Eric Wong <normalperson@yhbt.net> wrote:
Show 17 quoted lines
> Pascal Obry <pascal@obry.net> wrote:
> > I do not see something under .git/refs/* (only empty directories).
> >
> > The project has been imported using something like this:
> >
> >  $ git svn clone --prefix=svn/ svn+ssh://server/path \
> > 	--revision=580:HEAD \
> > 	--trunk=trunk/project \
> > 	--tags=tags/project \
> > 	--branches=branches/project \
> > 	--branches="branches/global/*/project" project
> 
> It looks like you didn't use any of the weird[1] options that make it
> unsafe to clobber the rev_maps.  Can you try removing all the .rev_map.*
> files in $GIT_DIR/svn and running "git svn fetch" to rebuild them?
> 
> [1] --no-metadata, --use-svm-props
Also, any chance you have multiple refs with "trunk" in the basename?
 git rev-parse --symbolic --all | grep '/trunk'

It could be a backwards compatibility issue with git svn looking in multiple places for trunk.

With Adam's commit, it'll try $GIT_DIR/svn/refs/remotes/trunk/* first Then it'll try $GIT_DIR/svn/trunk/* as a fallback.

-- 
Eric Wong
Pascal Obry· Nov 11, 2009, 21:55 UTC · re: Eric Wong · lore

Re: git-svn problem with v1.6.5

Eric,
> Also, any chance you have multiple refs with "trunk" in the basename?
>
>   git rev-parse --symbolic --all | grep '/trunk'
This reports:

$ git rev-parse --symbolic --all | grep '/trunk' refs/remotes/svn/trunk

> It could be a backwards compatibility issue with git svn looking
> in multiple places for trunk.
But I have multiple trunk:

$ ls .git/svn/svn/trunk/ ./ ../ .rev_map.936e1b1b-40f2-da11-902a-00137254ae57 unhandled.log

$ ls .git/svn/refs/remotes/svn/trunk/ ./ ../ .rev_map.936e1b1b-40f2-da11-902a-00137254ae57 unhandled.log

Removing all the .rev_map* fix the problem.

Removing only the one in .git/svn/refs/remotes/svn/trunk/ fix the problem too.

> With Adam's commit, it'll try $GIT_DIR/svn/refs/remotes/trunk/* first
> Then it'll try $GIT_DIR/svn/trunk/* as a fallback.

Does this means that it was looking first in $GIT_DIR/svn/trunk/* before? And the confusion come because now it is looking in $GIT_DIR/svn/refs/remotes/trunk/* first?

Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B
Eric Wong· Nov 11, 2009, 22:44 UTC · re: Pascal Obry · lore

Re: git-svn problem with v1.6.5

Pascal Obry <pascal@obry.net> wrote:
Show 10 quoted lines
> Eric,
>
>> Also, any chance you have multiple refs with "trunk" in the basename?
>>
>>   git rev-parse --symbolic --all | grep '/trunk'
>
> This reports:
>
> $ git rev-parse --symbolic --all | grep '/trunk'
> refs/remotes/svn/trunk
Pascal,
OK, this is normal.
Show 22 quoted lines
>> It could be a backwards compatibility issue with git svn looking
>> in multiple places for trunk.
>
> But I have multiple trunk:
>
> $ ls .git/svn/svn/trunk/
> ./  ../  .rev_map.936e1b1b-40f2-da11-902a-00137254ae57  unhandled.log
>
> $ ls  .git/svn/refs/remotes/svn/trunk/
> ./  ../  .rev_map.936e1b1b-40f2-da11-902a-00137254ae57  unhandled.log
>
> Removing all the .rev_map* fix the problem.
>
> Removing only the one in .git/svn/refs/remotes/svn/trunk/ fix the  
> problem too.
>
>> With Adam's commit, it'll try $GIT_DIR/svn/refs/remotes/trunk/* first
>> Then it'll try $GIT_DIR/svn/trunk/* as a fallback.
>
> Does this means that it was looking first in $GIT_DIR/svn/trunk/*  
> before? And the confusion come because now it is looking in  
> $GIT_DIR/svn/refs/remotes/trunk/* first?

Yes, but somehow $GIT_DIR/svn/refs/remotes/trunk should not have been created since $GIT_DIR/svn/svn/trunk already existed. Both of those directories existing at the same time should not

Did you try an early version of Adam's patch before it made it into git.git by any chance?

Or, did you by any chance start a fresh import with a v1.6.5 and then rsync $GIT_DIR to one created with 1.6.4 and not use --delete with rsync?

-- 
Eric Wong
Pascal Obry· Nov 12, 2009, 06:39 UTC · re: Eric Wong · lore

Re: git-svn problem with v1.6.5

Eric,
> Did you try an early version of Adam's patch before it made it into
> git.git by any chance?
No.
> Or, did you by any chance start a fresh import with a v1.6.5 and then
> rsync $GIT_DIR to one created with 1.6.4 and not use --delete with
> rsync?

No. The import has been done with 1.6.4. A filter-branch has been applied to fix author names. But AFAIK (will check) it has been done with 1.6.4 too.

Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B
Eric Wong· Nov 12, 2009, 07:11 UTC · re: Pascal Obry · lore

Re: git-svn problem with v1.6.5

Pascal Obry <pascal@obry.net> wrote:
Show 14 quoted lines
> Eric,
>
>> Did you try an early version of Adam's patch before it made it into
>> git.git by any chance?
>
> No.
>
>> Or, did you by any chance start a fresh import with a v1.6.5 and then
>> rsync $GIT_DIR to one created with 1.6.4 and not use --delete with
>> rsync?
>
> No. The import has been done with 1.6.4. A filter-branch has been  
> applied to fix author names. But AFAIK (will check) it has been done  
> with 1.6.4 too.

Oh... Any chance that lost commit you couldn't find was clobbered by filter-branch? That would explain a lot...

-- 
Eric Wong
Pascal Obry· Nov 12, 2009, 08:11 UTC · re: Eric Wong · lore

Re: git-svn problem with v1.6.5

Eric,
> Oh...  Any chance that lost commit you couldn't find was clobbered
> by filter-branch?  That would explain a lot...

I still have the AWS repo as it was from initial import. And I can see the commit on this one:

$ git log --all --parents | grep d2cf08bb67e4b7da33a250127aab784f1f2f58d3 commit bd95b2b254286e6f04015b7f58266d66abc134a7 d2cf08bb67e4b7da33a250127aab784f1f2f58d3 commit d2cf08bb67e4b7da33a250127aab784f1f2f58d3 406eaf7e51229f60048ea80cdb6904e33ef59a81

So that's definitely a problem with the filter-branch. Thanks a lot for your precious help!

Will review the procedure for filtering branches :(
Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595
Pascal Obry· Nov 12, 2009, 09:03 UTC · re: Pascal Obry · lore

Re: git-svn problem with v1.6.5

Eric,
> Will review the procedure for filtering branches :(
A quick review does not show something obviously wrong on my side.

The commit on the rewritten repo has a different sha-1 which is expected. So to me it seems that after a filter-branch on a git-svn repo it is mandatory to recreate all .rev_map*? Is that right? Is so, this is the missing piece in the procedure on my side.

Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595
Eric Wong· Nov 12, 2009, 09:35 UTC · re: Pascal Obry · lore

Re: git-svn problem with v1.6.5

Pascal Obry <pascal@obry.net> wrote:
Show 10 quoted lines
> Eric,
> 
> > Will review the procedure for filtering branches :(
> 
> A quick review does not show something obviously wrong on my side.
> 
> The commit on the rewritten repo has a different sha-1 which is
> expected. So to me it seems that after a filter-branch on a git-svn
> repo it is mandatory to recreate all .rev_map*? Is that right? Is so,
> this is the missing piece in the procedure on my side.

Yes, you need to recreate all the .rev_map* files if you use filter-branch. Just remove them and they should be automatically recreated for your case (not using --no-metadata or --use-svm-props)

-- 
Eric Wong
Pascal Obry· Nov 12, 2009, 10:00 UTC · re: Eric Wong · lore

Re: git-svn problem with v1.6.5

Eric,
> Yes, you need to recreate all the .rev_map* files if you use
> filter-branch.  Just remove them and they should be automatically
> recreated for your case (not using --no-metadata or --use-svm-props)
Thanks for the confirmation, I'll do that.
Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|              http://www.obry.net
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver wwwkeys.pgp.net --recv-key C1082595
Sam Vilain· Nov 11, 2009, 04:02 UTC · re: Pascal Obry · lore

Re: git-svn problem with v1.6.5

Pascal Obry wrote:
Show 14 quoted lines
> Hello,
>
> A git-svn mirror is created using Git v1.6.4. When I copy this mirror 
> into another machine which is using v1.6.5 I get the following error:
>
> $ git svn rebase
> fatal: Invalid revision range 
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk
> rev-list --pretty=raw --no-color --reverse 
> d2cf08bb67e4b7da33a250127aab784f1f2f58d3..refs/remotes/svn/trunk --: 
> command returned error: 128
>
> Using v1.6.4 instead I have no problem. Any idea to track this problem down?
>

Did you copy the entire directory across using rsync/tar/etc or 'git clone' ? You'll need to make sure that the 'refs/remotes/*' refs look exactly the same on the clone as they did on the original import repository.

Sam
Pascal Obry· Nov 11, 2009, 08:11 UTC · re: Sam Vilain · lore

Re: git-svn problem with v1.6.5

Sam,
> Did you copy the entire directory across using rsync/tar/etc or 'git
> clone' ?
I did a rsync of the .git directory so I expect all refs to be there.
Pascal.
-- 
--|------------------------------------------------------
--| Pascal Obry                           Team-Ada Member
--| 45, rue Gabriel Peri - 78114 Magny Les Hameaux FRANCE
--|------------------------------------------------------
--|    http://www.obry.net  -  http://v2p.fr.eu.org
--| "The best way to travel is by means of imagination"
--|
--| gpg --keyserver keys.gnupg.net --recv-key F949BD3B

← back to recent threads