git-svn does not support intermediate directories?

Hey all,

  After spending some hours struggling with git svn, it would appear
that it does not support svn projects stored in paths similar to
"http://foo.com/svn/repos/bar/myproject", where "myproject" uses the
standard SVN tags/trunk/branches layout.  I'm currently using git
1.6.1, though I tried this with 1.6.2-rc2 as well.  The resulting
.git/config looks something like this:

[svn-remote "svn"]
	url = http://foo.com/svn/repos/bar
	fetch = myproject/trunk:refs/remotes/trunk
	branches = bar/myproject/branches/*:refs/remotes/*
	tags = bar/myproject/tags/*:refs/remotes/tags/*

> 	$self->{path_regex} ||= qr/\/\Q$self->{path}\E\//;

Re: git-svn does not support intermediate directories?

Michael Lai(myllai@gmail.com)@030309-17:43:

>   After spending some hours struggling with git svn, it would appear
> that it does not support svn projects stored in paths similar to
> "http://foo.com/svn/repos/bar/myproject", where "myproject" uses the
> standard SVN tags/trunk/branches layout.  I'm currently using git
> 1.6.1, though I tried this with 1.6.2-rc2 as well.  The resulting
> .git/config looks something like this:
> 
> [svn-remote "svn"]
> 	url = http://foo.com/svn/repos/bar
> 	fetch = myproject/trunk:refs/remotes/trunk
> 	branches = bar/myproject/branches/*:refs/remotes/*
> 	tags = bar/myproject/tags/*:refs/remotes/tags/*
> 
> Yes, that's a redundant "bar" directory under "branches =" and "tags
> =".  The issue seems to lie in git-svn doing something intelligent to
> extract the appropriate trunk directory.  For the branches and tags,
> however, it just takes the full URL and removes the repository root
> (http://foo.com/svn/repos/bar) to produce "bar/myproject/{branches,
> tags}/*".  The second effect is that "git svn fetch" will run but exit
> quietly without actually pulling anything from the repository.  I
> tracked down an existing thread on the mailing list from a while ago
> (Feb 4th, title of "git-svn doesn't fetch anything"), but there was no
> resolution.

I've just run into this exact same issue.

> There is a quick workaround, which was to make this change to match_paths:
> < 	$self->{path_regex} ||= qr/^\/\Q$self->{path}\E\//;
> ---
> > 	$self->{path_regex} ||= qr/\/\Q$self->{path}\E\//;
> 
> The additional "bar" directory gets pulled in when git-svn tries to
> determine what paths to pull down, and tries to match
> "/myproject/trunk" to "/bar/myproject/trunk".  I've merely put a
> band-aid on the situation.  My perl is rudimentary at best, or I'd
> have spent additional time to try to put in a "proper" patch, but was
> wondering if anyone else had run into this problem and would be
> willing to put in a fix (or point me in the right direction, that
> works too).

I'd like a nicer solution too.

Tim

-- 
Tim Stoakes

Re: [RFC PATCH] git-svn does not support intermediate directories?

diff --git a/git-svn.perl b/git-svn.perl
index 959eb52..174f266 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -2351,7 +2351,11 @@ sub match_paths {
        if (my $path = $paths->{"/$self->{path}"}) {
                return ($path->{action} eq 'D') ? 0 : 1;
        }
-       $self->{path_regex} ||= qr/^\/\Q$self->{path}\E\//;
+       my $repos_root = $self->ra->{repos_root};
+       my $intermediate_path = $self->{url};
+       $intermediate_path =~ s#^\Q$repos_root\E(/|$)#\/#;
+       $intermediate_path .= '/' if $intermediate_path;
+       $self->{path_regex} ||= qr/^\/\Q$intermediate_path$self->{path}\E\//;
        if (grep /$self->{path_regex}/, keys %$paths) {
                return 1;
        }

--

On Tue, Mar 3, 2009 at 8:30 PM, Tim Stoakes <tim@stoakes.net> wrote:
> Michael Lai(myllai@gmail.com)@030309-17:43:
>>   After spending some hours struggling with git svn, it would appear
>> that it does not support svn projects stored in paths similar to
>> "http://foo.com/svn/repos/bar/myproject", where "myproject" uses the
>> standard SVN tags/trunk/branches layout.  I'm currently using git
>> 1.6.1, though I tried this with 1.6.2-rc2 as well.  The resulting
>> .git/config looks something like this:
>>
>> [svn-remote "svn"]
>>       url = http://foo.com/svn/repos/bar
>>       fetch = myproject/trunk:refs/remotes/trunk
>>       branches = bar/myproject/branches/*:refs/remotes/*
>>       tags = bar/myproject/tags/*:refs/remotes/tags/*
>>
>> Yes, that's a redundant "bar" directory under "branches =" and "tags
>> =".  The issue seems to lie in git-svn doing something intelligent to
>> extract the appropriate trunk directory.  For the branches and tags,
>> however, it just takes the full URL and removes the repository root
>> (http://foo.com/svn/repos/bar) to produce "bar/myproject/{branches,
>> tags}/*".  The second effect is that "git svn fetch" will run but exit
>> quietly without actually pulling anything from the repository.  I
>> tracked down an existing thread on the mailing list from a while ago
>> (Feb 4th, title of "git-svn doesn't fetch anything"), but there was no
>> resolution.
>
> I've just run into this exact same issue.
>
>> There is a quick workaround, which was to make this change to match_paths:
>> <     $self->{path_regex} ||= qr/^\/\Q$self->{path}\E\//;
>> ---
>> >     $self->{path_regex} ||= qr/\/\Q$self->{path}\E\//;
>>
>> The additional "bar" directory gets pulled in when git-svn tries to
>> determine what paths to pull down, and tries to match
>> "/myproject/trunk" to "/bar/myproject/trunk".  I've merely put a
>> band-aid on the situation.  My perl is rudimentary at best, or I'd
>> have spent additional time to try to put in a "proper" patch, but was
>> wondering if anyone else had run into this problem and would be
>> willing to put in a fix (or point me in the right direction, that
>> works too).
>
> I messed about with disabling $Git::SVN::_minimize_url, but that seemed
> to break other things.
>
> Made worse for me was the fact that my 'bar' in the present was renamed
> from 'baz' in the past, so git-svn couldn't find it at r1. Very
> confusing!
>
> I'd like a nicer solution too.
>
> Tim
>
> --
> Tim Stoakes
>

Re: [RFC PATCH] git-svn does not support intermediate directories?

Michael Lai <myllai@gmail.com> wrote:

> I did some additional hacking and may have found a slightly cleaner
> way of at least fixing the problems with "git svn fetch".  The problem
> with the wrong paths being initialized for branches and tags is fairly
> minor (since you can just edit the config by hand), so I'll probably
> address that later, if I have time.  Here's the patch (I hope I'm
> doing this right):

Hi Michael,

From cddc7e5bde060eb963534156ae0daaf41c87c21a Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Sat, 7 Mar 2009 20:22:29 -0800
Subject: [PATCH] git-svn: support intermediate paths when matching tags/branches
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

For repositories laid out like the following:

> [svn-remote "svn"]
>       url = http://foo.com/svn/repos/bar
>       fetch = myproject/trunk:refs/remotes/trunk
>       branches = bar/myproject/branches/*:refs/remotes/*
>       tags = bar/myproject/tags/*:refs/remotes/tags/*

  * extraneous slash removed from $intermediate_path,
    this was causing tests to fail.

  * fixed a case where $intermediate_path could be "0" and
    considered false by Perl, preventing the necessary
    slash from being appended.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
---
 git-svn.perl |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index 959eb52..745dd03 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -2351,7 +2351,11 @@ sub match_paths {
 	if (my $path = $paths->{"/$self->{path}"}) {
 		return ($path->{action} eq 'D') ? 0 : 1;
 	}
-	$self->{path_regex} ||= qr/^\/\Q$self->{path}\E\//;
+	my $repos_root = $self->ra->{repos_root};
+	my $intermediate_path = $self->{url};
+	$intermediate_path =~ s#^\Q$repos_root\E(/|$)##;
+	$intermediate_path .= '/' if length($intermediate_path) > 0;
+	$self->{path_regex} ||= qr/^\/\Q$intermediate_path$self->{path}\E\//;
 	if (grep /$self->{path_regex}/, keys %$paths) {
 		return 1;
 	}

-- 
Eric Wong

Re: [RFC PATCH] git-svn does not support intermediate directories?

> Your patch was whitespace damaged and lacked a proposed commit message.
> Please read Documentation/SubmittingPatches next time.

Hey Eric,

>
> Anyhow, I fixed your patch up a bit.  Can you sign-off on it
> if its right to you or let me know if it's broken?  Thanks.

From ae38acf85cfc86c075578c1c3f3c204d91d1d1f4 Mon Sep 17 00:00:00 2001
From: Michael Lai <myllai@gmail.com>
Date: Mon, 9 Mar 2009 11:45:47 -0700
Subject: [PATCH] git-svn: support intermediate paths when matching tags/branches

For repositories laid out like the following:

[svn-remote "svn"]
      url = http://foo.com/svn/repos/bar
      fetch = myproject/trunk:refs/remotes/trunk
      branches = bar/myproject/branches/*:refs/remotes/*
      tags = bar/myproject/tags/*:refs/remotes/tags/*

Signed-off-by: Michael Lai <myllai@gmail.com>
---
 git-svn.perl |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/git-svn.perl b/git-svn.perl
index 959eb52..8be6be0 100755
--- a/git-svn.perl
+++ b/git-svn.perl
@@ -2351,7 +2351,10 @@ sub match_paths {
 	if (my $path = $paths->{"/$self->{path}"}) {
 		return ($path->{action} eq 'D') ? 0 : 1;
 	}
-	$self->{path_regex} ||= qr/^\/\Q$self->{path}\E\//;
+	my $repos_root = $self->ra->{repos_root};
+	my $extended_path = $self->{url} . '/' . $self->{path};
+	$extended_path =~ s#^\Q$repos_root\E(/|$)##;
+	$self->{path_regex} ||= qr/^\/\Q$extended_path\E\//;
 	if (grep /$self->{path_regex}/, keys %$paths) {
 		return 1;
 	}

-- 
1.6.2

>
> From cddc7e5bde060eb963534156ae0daaf41c87c21a Mon Sep 17 00:00:00 2001
> From: Eric Wong <normalperson@yhbt.net>
> Date: Sat, 7 Mar 2009 20:22:29 -0800
> Subject: [PATCH] git-svn: support intermediate paths when matching tags/branches
> MIME-Version: 1.0
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: 8bit
>
> For repositories laid out like the following:
>
>> [svn-remote "svn"]
>>       url = http://foo.com/svn/repos/bar
>>       fetch = myproject/trunk:refs/remotes/trunk
>>       branches = bar/myproject/branches/*:refs/remotes/*
>>       tags = bar/myproject/tags/*:refs/remotes/tags/*
>
> The "bar" component above is considered the intermediate path
> and was not handled correctly.
>
> This patch was originally by Michael Lai (without a commit
> message) with some minor fixes:
>
>  * extraneous slash removed from $intermediate_path,
>    this was causing tests to fail.
>
>  * fixed a case where $intermediate_path could be "0" and
>    considered false by Perl, preventing the necessary
>    slash from being appended.
>
> Signed-off-by: Eric Wong <normalperson@yhbt.net>
> ---
>  git-svn.perl |    6 +++++-
>  1 files changed, 5 insertions(+), 1 deletions(-)
>
> diff --git a/git-svn.perl b/git-svn.perl
> index 959eb52..745dd03 100755
> --- a/git-svn.perl
> +++ b/git-svn.perl
> @@ -2351,7 +2351,11 @@ sub match_paths {
>        if (my $path = $paths->{"/$self->{path}"}) {
>                return ($path->{action} eq 'D') ? 0 : 1;
>        }
> -       $self->{path_regex} ||= qr/^\/\Q$self->{path}\E\//;
> +       my $repos_root = $self->ra->{repos_root};
> +       my $intermediate_path = $self->{url};
> +       $intermediate_path =~ s#^\Q$repos_root\E(/|$)##;
> +       $intermediate_path .= '/' if length($intermediate_path) > 0;
> +       $self->{path_regex} ||= qr/^\/\Q$intermediate_path$self->{path}\E\//;
>        if (grep /$self->{path_regex}/, keys %$paths) {
>                return 1;
>        }
> --
> Eric Wong
>

Re: [RFC PATCH] git-svn does not support intermediate directories?

Michael Lai <myllai@gmail.com> wrote:

> > Your patch was whitespace damaged and lacked a proposed commit message.
> > Please read Documentation/SubmittingPatches next time.
> Hey Eric,
> 
> Sorry, I didn't notice that; I've read through it and hopefully my
> patches should conform from now on.
> 
> >
> > Anyhow, I fixed your patch up a bit.  Can you sign-off on it
> > if its right to you or let me know if it's broken?  Thanks.
> 
> I looked through the patch and that would work, but at the same time I
> had another idea which may be a little cleaner.  Let me know what you
> think.

Thanks Michael, looks good to me,

Acked and pushed out to git://git.bogomips.org/git-svn

> From ae38acf85cfc86c075578c1c3f3c204d91d1d1f4 Mon Sep 17 00:00:00 2001
> From: Michael Lai <myllai@gmail.com>
> Date: Mon, 9 Mar 2009 11:45:47 -0700
> Subject: [PATCH] git-svn: support intermediate paths when matching tags/branches
> 
> For repositories laid out like the following:
> 
> [svn-remote "svn"]
>       url = http://foo.com/svn/repos/bar
>       fetch = myproject/trunk:refs/remotes/trunk
>       branches = bar/myproject/branches/*:refs/remotes/*
>       tags = bar/myproject/tags/*:refs/remotes/tags/*
> 
> The "bar" component above is considered the intermediate path
> and was not handled correctly.
> 
> Signed-off-by: Michael Lai <myllai@gmail.com>
> ---
>  git-svn.perl |    5 ++++-
>  1 files changed, 4 insertions(+), 1 deletions(-)
> 
> diff --git a/git-svn.perl b/git-svn.perl
> index 959eb52..8be6be0 100755
> --- a/git-svn.perl
> +++ b/git-svn.perl
> @@ -2351,7 +2351,10 @@ sub match_paths {
>  	if (my $path = $paths->{"/$self->{path}"}) {
>  		return ($path->{action} eq 'D') ? 0 : 1;
>  	}
> -	$self->{path_regex} ||= qr/^\/\Q$self->{path}\E\//;
> +	my $repos_root = $self->ra->{repos_root};
> +	my $extended_path = $self->{url} . '/' . $self->{path};
> +	$extended_path =~ s#^\Q$repos_root\E(/|$)##;
> +	$self->{path_regex} ||= qr/^\/\Q$extended_path\E\//;
>  	if (grep /$self->{path_regex}/, keys %$paths) {
>  		return 1;
>  	}
> -- 
> 1.6.2