threads / patch / 6257

patchgit-svnimport: clean svn path when accessing SVN repo

Subject: [PATCH] git-svnimport: clean svn path when accessing SVN repo

## tl;dr

19 messages between Dec 7, 2006 and Jan 7, 2007. Diffs are folded; open one to read it.

replies: 18people: 4as markdown or json

Daniel Drake· Dec 7, 2006, 15:26 UTC · lore

git-svnimport breakage as of git-1.4.4

Hi,
git-svnimport broken between git-1.4.3.5 and git-1.4.4

I have found that commit 83936a29e275bc0c04f60d3333e4951a9e16b1fc is the cause of this.

I am using git-svnimport to work with a repo with this layout:

https://server/repo/trunk https://server/repo/tags/x.y.z https://server/repo/branches/somebranch

Starting a fresh import:
# git-svnimport -v -i -C repo -r https://server repo
Fetching from 1 to 10707 ...
Tree ID 4b825dc642cb6eb9a060e54bf8d69288fbee4904
Committed change 1:/ 2004-12-22 22:53:27)
Committing initial tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904
Commit ID 2614c05ac4c5f24eb89cea056a7d46c909084d8c
Writing to refs/heads/origin
DONE: 1 origin 2614c05ac4c5f24eb89cea056a7d46c909084d8c
RA layer request failed: PROPFIND request failed on '/trunk/.cvsignore':
PROPFIND of '/trunk/.cvsignore': 405 Method Not Allowed (https://server)
at /usr/bin/git-svnimport line 358

According to the server logs, git is requesting /trunk/.cvsignore rather than /repo/trunk/.cvsignore

I'm happy to test patches and whatnot but don't have time to investigate further right now.

Thanks!
-- 
Daniel Drake
Brontes Technologies, A 3M Company
Sasha Khapyorsky· Dec 8, 2006, 20:32 UTC · re: Daniel Drake · lore

Re: git-svnimport breakage as of git-1.4.4

Hi,
On 10:26 Thu 07 Dec     , Daniel Drake wrote:
Show 15 quoted lines
> 
> git-svnimport broken between git-1.4.3.5 and git-1.4.4
> 
> I have found that commit 83936a29e275bc0c04f60d3333e4951a9e16b1fc is the
> cause of this.
> 
> I am using git-svnimport to work with a repo with this layout:
> 
> https://server/repo/trunk
> https://server/repo/tags/x.y.z
> https://server/repo/branches/somebranch
> 
> Starting a fresh import:
> 
> # git-svnimport -v -i -C repo -r https://server repo
Is this 'server' public? Can I rerun this git-svnimport?
If not, please try the patch:
Show changes to git-svnimport.perl +7 −1
diff --git a/git-svnimport.perl b/git-svnimport.perl
index cbaa8ab..b9de446 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -210,6 +210,12 @@ $svn .= "/$svn_dir" if defined $svn_dir;
 my $svn2 = SVNconn->new($svn);
 $svn = SVNconn->new($svn);
 
+if($svn_dir) {
+	$svn_dir =~ s#/*$#/#;
+} else {
+	$svn_dir = "";
+}
+
 my $lwp_ua;
 if($opt_d or $opt_D) {
 	$svn_url = URI->new($svn_url)->canonical;
@@ -906,7 +912,7 @@ sub commit_all {
 	my ($changed_paths, $revision, $author, $date, $message, $pool) = @_;
 	my %p;
 	while(my($path,$action) = each %$changed_paths) {
-		$p{$path} = [ $action->action,$action->copyfrom_path, $action->copyfrom_rev, $path ];
+		$p{$path} = [ $action->action,$svn_dir$action->copyfrom_path, $action->copyfrom_rev, $svn_dir$path ];
 	}
 	$changed_paths = \%p;
 

Thanks,
Dongsheng Song· Dec 10, 2006, 03:49 UTC · re: Sasha Khapyorsky · lore

Re: git-svnimport breakage as of git-1.4.4

I met the broken too, when I downgrade to 1.4.3.4, it's fine.
I have not test your patch, but you can try your self,
http://tortoisesvn.tigris.org/svn/tortoisesvn
and the master branch(today) fail between r6000~r7000 too
2006/12/9, Sasha Khapyorsky <sashak@voltaire.com>:
Show 10 quoted lines
> Hi,
>
> On 10:26 Thu 07 Dec     , Daniel Drake wrote:
> >
> > git-svnimport broken between git-1.4.3.5 and git-1.4.4
> >
>
> Is this 'server' public? Can I rerun this git-svnimport?
>
> If not, please try the patch:
Sasha Khapyorsky· Dec 10, 2006, 11:47 UTC · re: Dongsheng Song · lore

Re: git-svnimport breakage as of git-1.4.4

On 11:49 Sun 10 Dec     , Dongsheng Song wrote:
Show 7 quoted lines
> I met the broken too, when I downgrade to 1.4.3.4, it's fine.
> 
> I have not test your patch, but you can try your self,
> 
> http://tortoisesvn.tigris.org/svn/tortoisesvn
> 
> and the master branch(today) fail between r6000~r7000 too

Thanks for the link. but I cannot access - this requires username/password authentication.

Dongsheng Song· Dec 11, 2006, 20:00 UTC · re: Sasha Khapyorsky · lore

Re: git-svnimport breakage as of git-1.4.4

Sorry, I assume you have see http://tortoisesvn.tigris.org/:

username : guest password : ""

2006/12/10, Sasha Khapyorsky <sashak@voltaire.com>:
Show 13 quoted lines
> On 11:49 Sun 10 Dec     , Dongsheng Song wrote:
> > I met the broken too, when I downgrade to 1.4.3.4, it's fine.
> >
> > I have not test your patch, but you can try your self,
> >
> > http://tortoisesvn.tigris.org/svn/tortoisesvn
> >
> > and the master branch(today) fail between r6000~r7000 too
>
> Thanks for the link. but I cannot access - this requires
> username/password authentication.
>
> Sasha
Sasha Khapyorsky· Dec 11, 2006, 20:50 UTC · re: Dongsheng Song · lore

Re: git-svnimport breakage as of git-1.4.4

On 04:00 Tue 12 Dec     , Dongsheng Song wrote:
> Sorry, I assume you have see http://tortoisesvn.tigris.org/:
> 
> username : guest
> password : ""
Thanks, I can grab svn log now. Which command line you are using?
Dongsheng Song· Dec 11, 2006, 21:01 UTC · re: Sasha Khapyorsky · lore

Re: git-svnimport breakage as of git-1.4.4

$ cat svn2git-tortoisesvn.sh #!/bin/sh

export LC_ALL=C export WC_ROOT=/home/cauchy/wc/git export REPO_NAME=tortoisesvn

echo "[`date`] Start import & pack ..." mr=0 while [ $mr -le 9000 ]; do

    if test -f $WC_ROOT/$REPO_NAME/.git/SVN2GIT_HEAD; then
        echo "[`date`] clean up ..."
        cd $WC_ROOT/$REPO_NAME
        git-read-tree -m -u SVN2GIT_HEAD HEAD && rm -f .git/SVN2GIT_HEAD
        echo "[`date`] clean up finished"
    fi
    mr=$(($mr + 1000))
    echo "[`date`] Start import up to revison $mr ..."
    git-svnimport -v -i -r -o master -l $mr -C $WC_ROOT/$REPO_NAME
http://tortoisesvn.tigris.org/svn/tortoisesvn
    echo "[`date`] Finish import up to revison $mr"
    cd $WC_ROOT/$REPO_NAME && git-repack -a -d --window=64 --depth=64
    echo "[`date`] Finish repack revison $mr"
    cd $WC_ROOT/$REPO_NAME && find .git -name pack | xargs ls -l
done
echo "[`date`] Finished import & pack"
2006/12/12, Sasha Khapyorsky <sashak@voltaire.com>:
Show 9 quoted lines
> On 04:00 Tue 12 Dec     , Dongsheng Song wrote:
> > Sorry, I assume you have see http://tortoisesvn.tigris.org/:
> >
> > username : guest
> > password : ""
>
> Thanks, I can grab svn log now. Which command line you are using?
>
> Sasha
Sasha Khapyorsky· Dec 14, 2006, 02:25 UTC · re: Dongsheng Song · lore

Re: git-svnimport breakage as of git-1.4.4

On 05:01 Tue 12 Dec     , Dongsheng Song wrote:
> 
>    git-svnimport -v -i -r -o master -l $mr -C $WC_ROOT/$REPO_NAME
> http://tortoisesvn.tigris.org/svn/tortoisesvn
Thanks.

I'm running now git-svnimport against http://tortoisesvn.tigris.org/svn/tortoisesvn with follow patch:

Show changes to git-svnimport.perl +4 −0
diff --git a/git-svnimport.perl b/git-svnimport.perl
index cbaa8ab..071777b 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -146,6 +146,7 @@ sub file {
 	print "... $rev $path ...\n" if $opt_v;
 	my (undef, $properties);
 	my $pool = SVN::Pool->new();
+	$path =~ s#^/*##;
 	eval { (undef, $properties)
 		   = $self->{'svn'}->get_file($path,$rev,$fh,$pool); };
 	$pool->clear;
@@ -181,6 +182,7 @@ sub ignore {
 	my($self,$path,$rev) = @_;
 
 	print "... $rev $path ...\n" if $opt_v;
+	$path =~ s#^/*##;
 	my (undef,undef,$properties)
 	    = $self->{'svn'}->get_dir($path,$rev,undef);
 	if (exists $properties->{'svn:ignore'}) {
@@ -197,6 +199,7 @@ sub ignore {
 
 sub dir_list {
 	my($self,$path,$rev) = @_;
+	$path =~ s#^/*##;
 	my ($dirents,undef,$properties)
 	    = $self->{'svn'}->get_dir($path,$rev,undef);
 	return $dirents;
@@ -354,6 +357,7 @@ open BRANCHES,">>", "$git_dir/svn2git";
 sub node_kind($$) {
 	my ($svnpath, $revision) = @_;
 	my $pool=SVN::Pool->new;
+	$svnpath =~ s#^/*##;
 	my $kind = $svn->{'svn'}->check_path($svnpath,$revision,$pool);
 	$pool->clear;
 	return $kind;


And it works up to now.

This is the same patch as recently posted to Daniel. Could you try?
Daniel Drake· Dec 11, 2006, 14:27 UTC · re: Sasha Khapyorsky · lore

Re: git-svnimport breakage as of git-1.4.4

On Fri, 2006-12-08 at 22:32 +0200, Sasha Khapyorsky wrote:
> > # git-svnimport -v -i -C repo -r https://server repo
> 
> Is this 'server' public? Can I rerun this git-svnimport?
Sorry, it is not.
Show 6 quoted lines
> @@ -906,7 +912,7 @@ sub commit_all {
>  	my ($changed_paths, $revision, $author, $date, $message, $pool) = @_;
>  	my %p;
>  	while(my($path,$action) = each %$changed_paths) {
> -		$p{$path} = [ $action->action,$action->copyfrom_path, $action->copyfrom_rev, $path ];
> +		$p{$path} = [ $action->action,$svn_dir$action->copyfrom_path, $action->copyfrom_rev, $svn_dir$path ];
This is not valid perl - I think you wanted $svn_dir . $path
After making that modification it's not fixed though:
Fetching from 1 to 10742 ...
Tree ID 4b825dc642cb6eb9a060e54bf8d69288fbee4904
Committed change 1:/ 2004-12-22 22:53:27)
Committing initial tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904
Commit ID 2614c05ac4c5f24eb89cea056a7d46c909084d8c
Writing to refs/heads/origin
DONE: 1 origin 2614c05ac4c5f24eb89cea056a7d46c909084d8c
perl: subversion/libsvn_subr/path.c:377: svn_path_basename: Assertion
`is_canonical (path, len)' failed.
Aborted
-- 
Daniel Drake
Brontes Technologies, A 3M Company
Sasha Khapyorsky· Dec 11, 2006, 20:49 UTC · re: Daniel Drake · lore

Re: git-svnimport breakage as of git-1.4.4

On 09:27 Mon 11 Dec     , Daniel Drake wrote:
Show 15 quoted lines
> On Fri, 2006-12-08 at 22:32 +0200, Sasha Khapyorsky wrote:
> > > # git-svnimport -v -i -C repo -r https://server repo
> > 
> > Is this 'server' public? Can I rerun this git-svnimport?
> 
> Sorry, it is not.
> 
> > @@ -906,7 +912,7 @@ sub commit_all {
> >  	my ($changed_paths, $revision, $author, $date, $message, $pool) = @_;
> >  	my %p;
> >  	while(my($path,$action) = each %$changed_paths) {
> > -		$p{$path} = [ $action->action,$action->copyfrom_path, $action->copyfrom_rev, $path ];
> > +		$p{$path} = [ $action->action,$svn_dir$action->copyfrom_path, $action->copyfrom_rev, $svn_dir$path ];
> 
> This is not valid perl - I think you wanted $svn_dir . $path
Yes, sorry.
Show 13 quoted lines
> 
> After making that modification it's not fixed though:
> 
> Fetching from 1 to 10742 ...
> Tree ID 4b825dc642cb6eb9a060e54bf8d69288fbee4904
> Committed change 1:/ 2004-12-22 22:53:27)
> Committing initial tree 4b825dc642cb6eb9a060e54bf8d69288fbee4904
> Commit ID 2614c05ac4c5f24eb89cea056a7d46c909084d8c
> Writing to refs/heads/origin
> DONE: 1 origin 2614c05ac4c5f24eb89cea056a7d46c909084d8c
> perl: subversion/libsvn_subr/path.c:377: svn_path_basename: Assertion
> `is_canonical (path, len)' failed.
> Aborted

Maybe I'm starting to understand. Your svn url (url which points to svn repository) is https://server/repo and not just https://server, right?

If so, please remove the patch (you don't need it) and rerun:
  git-svnimport -v -i -C repo -r https://server/repo
Daniel Drake· Dec 11, 2006, 21:03 UTC · re: Sasha Khapyorsky · lore

Re: git-svnimport breakage as of git-1.4.4

On Mon, 2006-12-11 at 22:49 +0200, Sasha Khapyorsky wrote:
> Maybe I'm starting to understand. Your svn url (url which points to svn
> repository) is https://server/repo and not just https://server, right?
Yes, and then under that we have https://server/repo/trunk
> If so, please remove the patch (you don't need it) and rerun:
> 
>   git-svnimport -v -i -C repo -r https://server/repo

Ah, that fixes it. However, in versions before 1.4.4, either invokation style works.

Thanks,
-- 
Daniel Drake
Brontes Technologies, A 3M Company
Sasha Khapyorsky· Dec 11, 2006, 22:03 UTC · re: Daniel Drake · lore

Re: git-svnimport breakage as of git-1.4.4

On 16:03 Mon 11 Dec     , Daniel Drake wrote:
Show 12 quoted lines
> On Mon, 2006-12-11 at 22:49 +0200, Sasha Khapyorsky wrote:
> > Maybe I'm starting to understand. Your svn url (url which points to svn
> > repository) is https://server/repo and not just https://server, right?
> 
> Yes, and then under that we have https://server/repo/trunk
> 
> > If so, please remove the patch (you don't need it) and rerun:
> > 
> >   git-svnimport -v -i -C repo -r https://server/repo
> 
> Ah, that fixes it. However, in versions before 1.4.4, either invokation
> style works.
Frankly I think that it was bug. And I will see how to restore this. :)
Daniel Drake· Dec 13, 2006, 16:28 UTC · re: Sasha Khapyorsky · lore

Re: git-svnimport breakage as of git-1.4.4

On Mon, 2006-12-11 at 22:49 +0200, Sasha Khapyorsky wrote:
Show 6 quoted lines
> Maybe I'm starting to understand. Your svn url (url which points to svn
> repository) is https://server/repo and not just https://server, right?
> 
> If so, please remove the patch (you don't need it) and rerun:
> 
>   git-svnimport -v -i -C repo -r https://server/repo

Sorry, apparently I was using the wrong git-svnimport in my last mail. The above command, with or without your svn_dir patch, doesn't solve the problem.

With your patch:
# git-svnimport -v -i -C repo -r https://server/repo

RA layer request failed: PROPFIND request failed on '/trunk/.cvsignore': PROPFIND of '/trunk/.cvsignore': 405 Method Not Allowed (https://svn) at git-svnimport line 364

# git-svnimport -v -i -C repo -r https://server repo perl: subversion/libsvn_subr/path.c:377: svn_path_basename: Assertion `is_canonical (path, len)' failed. Aborted

Without the patch, the error is the same as the 1st case in both situations.

-- 
Daniel Drake
Brontes Technologies, A 3M Company
Sasha Khapyorsky· Dec 14, 2006, 02:21 UTC · re: Daniel Drake · lore

Re: git-svnimport breakage as of git-1.4.4

On 11:28 Wed 13 Dec     , Daniel Drake wrote:
Show 6 quoted lines
> 
> Sorry, apparently I was using the wrong git-svnimport in my last mail.
> The above command, with or without your svn_dir patch, doesn't solve the
> problem.
> 
> With your patch:
Original patch is wrong, so only w/out this patch.
Show 9 quoted lines
> 
> # git-svnimport -v -i -C repo -r https://server/repo
> 
> RA layer request failed: PROPFIND request failed on '/trunk/.cvsignore':
> PROPFIND of '/trunk/.cvsignore': 405 Method Not Allowed (https://svn) at
> git-svnimport line 364
> 
> Without the patch, the error is the same as the 1st case in both
> situations.
Try this please:
Show changes to git-svnimport.perl +4 −0
diff --git a/git-svnimport.perl b/git-svnimport.perl
index cbaa8ab..071777b 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -146,6 +146,7 @@ sub file {
 	print "... $rev $path ...\n" if $opt_v;
 	my (undef, $properties);
 	my $pool = SVN::Pool->new();
+	$path =~ s#^/*##;
 	eval { (undef, $properties)
 		   = $self->{'svn'}->get_file($path,$rev,$fh,$pool); };
 	$pool->clear;
@@ -181,6 +182,7 @@ sub ignore {
 	my($self,$path,$rev) = @_;
 
 	print "... $rev $path ...\n" if $opt_v;
+	$path =~ s#^/*##;
 	my (undef,undef,$properties)
 	    = $self->{'svn'}->get_dir($path,$rev,undef);
 	if (exists $properties->{'svn:ignore'}) {
@@ -197,6 +199,7 @@ sub ignore {
 
 sub dir_list {
 	my($self,$path,$rev) = @_;
+	$path =~ s#^/*##;
 	my ($dirents,undef,$properties)
 	    = $self->{'svn'}->get_dir($path,$rev,undef);
 	return $dirents;
@@ -354,6 +357,7 @@ open BRANCHES,">>", "$git_dir/svn2git";
 sub node_kind($$) {
 	my ($svnpath, $revision) = @_;
 	my $pool=SVN::Pool->new;
+	$svnpath =~ s#^/*##;
 	my $kind = $svn->{'svn'}->check_path($svnpath,$revision,$pool);
 	$pool->clear;
 	return $kind;

Thanks,
Daniel Drake· Dec 14, 2006, 21:05 UTC · re: Sasha Khapyorsky · lore

Re: git-svnimport breakage as of git-1.4.4

On Thu, 2006-12-14 at 04:21 +0200, Sasha Khapyorsky wrote:
Show 7 quoted lines
> Try this please:
> 
> 
> diff --git a/git-svnimport.perl b/git-svnimport.perl
> index cbaa8ab..071777b 100755
> --- a/git-svnimport.perl
> +++ b/git-svnimport.perl
Thanks, it now works for both forms of command line arguments.
-- 
Daniel Drake
Brontes Technologies, A 3M Company
Sasha Khapyorsky· Dec 14, 2006, 21:20 UTC · re: Daniel Drake · lore

Re: git-svnimport breakage as of git-1.4.4

On 16:05 Thu 14 Dec     , Daniel Drake wrote:
Show 10 quoted lines
> On Thu, 2006-12-14 at 04:21 +0200, Sasha Khapyorsky wrote:
> > Try this please:
> > 
> > 
> > diff --git a/git-svnimport.perl b/git-svnimport.perl
> > index cbaa8ab..071777b 100755
> > --- a/git-svnimport.perl
> > +++ b/git-svnimport.perl
> 
> Thanks, it now works for both forms of command line arguments.

Thanks for reporting. I still run git-svnimport against http://tortoisesvn.tigris.org/svn/tortoisesvn, works fine up to now.

Junio C Hamano· Dec 14, 2006, 21:32 UTC · re: Sasha Khapyorsky · lore

Re: git-svnimport breakage as of git-1.4.4

Sasha Khapyorsky <sashak@voltaire.com> writes:
Show 14 quoted lines
> On 16:05 Thu 14 Dec     , Daniel Drake wrote:
>> On Thu, 2006-12-14 at 04:21 +0200, Sasha Khapyorsky wrote:
>> > Try this please:
>> > 
>> > 
>> > diff --git a/git-svnimport.perl b/git-svnimport.perl
>> > index cbaa8ab..071777b 100755
>> > --- a/git-svnimport.perl
>> > +++ b/git-svnimport.perl
>> 
>> Thanks, it now works for both forms of command line arguments.
>
> Thanks for reporting. I still run git-svnimport against
> http://tortoisesvn.tigris.org/svn/tortoisesvn, works fine up to now.

An applicable version of the patch with proposed commit log message would be much appreciated.

Sasha Khapyorsky· Dec 14, 2006, 21:43 UTC · re: Junio C Hamano · lore

Re: git-svnimport breakage as of git-1.4.4

On 13:32 Thu 14 Dec     , Junio C Hamano wrote:
Show 6 quoted lines
> >
> > Thanks for reporting. I still run git-svnimport against
> > http://tortoisesvn.tigris.org/svn/tortoisesvn, works fine up to now.
> 
> An applicable version of the patch with proposed commit log
> message would be much appreciated.
Sure.

Wanted at least to finish the test (it is running yet), then will submit the patch in conventional way.

Sasha Khapyorsky· Jan 7, 2007, 00:22 UTC · re: Sasha Khapyorsky · lore
Clean svn path from leading '/' when accessing SVN repo.
Signed-off-by: Sasha Khapyorsky <sashak@voltaire.com>
---

This fixes git-svnimport problems reported in this thread ("git-svnimport breakage as of git-1.4.4"). Finally I forgot to submit this then, sorry about that.

 git-svnimport.perl |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
Show changes to git-svnimport.perl +4 −0
diff --git a/git-svnimport.perl b/git-svnimport.perl
index cbaa8ab..071777b 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -146,6 +146,7 @@ sub file {
 	print "... $rev $path ...\n" if $opt_v;
 	my (undef, $properties);
 	my $pool = SVN::Pool->new();
+	$path =~ s#^/*##;
 	eval { (undef, $properties)
 		   = $self->{'svn'}->get_file($path,$rev,$fh,$pool); };
 	$pool->clear;
@@ -181,6 +182,7 @@ sub ignore {
 	my($self,$path,$rev) = @_;
 
 	print "... $rev $path ...\n" if $opt_v;
+	$path =~ s#^/*##;
 	my (undef,undef,$properties)
 	    = $self->{'svn'}->get_dir($path,$rev,undef);
 	if (exists $properties->{'svn:ignore'}) {
@@ -197,6 +199,7 @@ sub ignore {
 
 sub dir_list {
 	my($self,$path,$rev) = @_;
+	$path =~ s#^/*##;
 	my ($dirents,undef,$properties)
 	    = $self->{'svn'}->get_dir($path,$rev,undef);
 	return $dirents;
@@ -354,6 +357,7 @@ open BRANCHES,">>", "$git_dir/svn2git";
 sub node_kind($$) {
 	my ($svnpath, $revision) = @_;
 	my $pool=SVN::Pool->new;
+	$svnpath =~ s#^/*##;
 	my $kind = $svn->{'svn'}->check_path($svnpath,$revision,$pool);
 	$pool->clear;
 	return $kind;

-- 
1.5.0.rc0.g2484-dirty

← back to recent threads