{"thread":{"id":"28618","subject":"[PATCH] git-svn: Allow certain refs to be ignored","startedAt":"2011-10-07T00:41:20Z","lastAt":"2011-10-14T09:24:57Z","messageCount":5,"participants":["Michael Olson","Junio C Hamano","Eric Wong","Piotr Krukowiecki"],"isPatch":true,"patchVersion":1,"patchTotal":null},"messages":[{"id":"177126","messageId":"CAN4ruPiSgY+LPdDgS021WQyoHMuNrJDzrqMuCt9G5qfZ=XtjoQ@mail.gmail.com","threadId":"28618","inReplyTo":null,"subject":"[PATCH] git-svn: Allow certain refs to be ignored","fromName":"Michael Olson","fromEmail":"mwolson@gnu.org","sentAt":"2011-10-07T00:41:20Z","receivedAt":"2011-10-07T00:41:20Z","isPatch":true,"sender":{"key":"mwolson@gnu.org","avatar":"https://avatars.githubusercontent.com/u/1510193?v=4"},"body":"Implement a new --ignore-refs option which specifies a regex of refs\nto ignore while importing svn history.\n\nThis is a useful supplement to the --ignore-paths option, as that\noption only operates on the contents of branches and tags, not the\nbranches and tags themselves.\n\nSigned-off-by: Michael Olson <mwolson@gnu.org>\n---\nRe-sent by request of Piotr Krukowiecki.  This is against v1.7.4.1,\nand I've been using it stably for a while.\n\n git-svn.perl |   38 +++++++++++++++++++++++++++++++++-----\n 1 files changed, 33 insertions(+), 5 deletions(-)\n\ndiff --git a/git-svn.perl b/git-svn.perl\nindex 177dd25..541fa2d 100755\n--- a/git-svn.perl\n+++ b/git-svn.perl\n@@ -90,7 +90,8 @@ $_q ||= 0;\n my %remote_opts = ( 'username=s' => \\$Git::SVN::Prompt::_username,\n                     'config-dir=s' => \\$Git::SVN::Ra::config_dir,\n                     'no-auth-cache' => \\$Git::SVN::Prompt::_no_auth_cache,\n-                    'ignore-paths=s' => \\$SVN::Git::Fetcher::_ignore_regex );\n+                    'ignore-paths=s' => \\$SVN::Git::Fetcher::_ignore_regex,\n+                    'ignore-refs=s' => \\$Git::SVN::Ra::_ignore_refs_regex );\n my %fc_opts = ( 'follow-parent|follow!' => \\$Git::SVN::_follow_parent,\n \t\t'authors-file|A=s' => \\$_authors,\n \t\t'authors-prog=s' => \\$_authors_prog,\n@@ -380,9 +381,12 @@ sub do_git_init_db {\n \t\tcommand_noisy('config', \"$pfx.$i\", $icv{$i});\n \t\t$set = $i;\n \t}\n-\tmy $ignore_regex = \\$SVN::Git::Fetcher::_ignore_regex;\n-\tcommand_noisy('config', \"$pfx.ignore-paths\", $$ignore_regex)\n-\t\tif defined $$ignore_regex;\n+\tmy $ignore_paths_regex = \\$SVN::Git::Fetcher::_ignore_regex;\n+\tcommand_noisy('config', \"$pfx.ignore-paths\", $$ignore_paths_regex)\n+\t\tif defined $$ignore_paths_regex;\n+\tmy $ignore_refs_regex = \\$Git::SVN::Ra::_ignore_refs_regex;\n+\tcommand_noisy('config', \"$pfx.ignore-refs\", $$ignore_refs_regex)\n+\t\tif defined $$ignore_refs_regex;\n }\n\n sub init_subdir {\n@@ -1831,6 +1835,8 @@ sub read_all_remotes {\n \t\t\t$r->{$1}->{svm} = {};\n \t\t} elsif (m!^(.+)\\.url=\\s*(.*)\\s*$!) {\n \t\t\t$r->{$1}->{url} = $2;\n+\t\t} elsif (m!^(.+)\\.ignore-refs=\\s*(.*)\\s*$!) {\n+\t\t\t$r->{$1}->{ignore_refs_regex} = $2;\n \t\t} elsif (m!^(.+)\\.(branches|tags)=$svn_refspec$!) {\n \t\t\tmy ($remote, $t, $local_ref, $remote_ref) =\n \t\t\t                                     ($1, $2, $3, $4);\n@@ -1867,6 +1873,16 @@ sub read_all_remotes {\n \t\t}\n \t} keys %$r;\n\n+\tforeach my $remote (keys %$r) {\n+\t\tforeach ( grep { defined $_ }\n+\t\t\t  map { $r->{$remote}->{$_} } qw(branches tags) ) {\n+\t\t\tforeach my $rs ( @$_ ) {\n+\t\t\t\t$rs->{ignore_refs_regex} =\n+\t\t\t\t    $r->{$remote}->{ignore_refs_regex};\n+\t\t\t}\n+\t\t}\n+\t}\n+\n \t$r;\n }\n\n@@ -4876,7 +4892,7 @@ sub apply_diff {\n }\n\n package Git::SVN::Ra;\n-use vars qw/@ISA $config_dir $_log_window_size/;\n+use vars qw/@ISA $config_dir $_ignore_refs_regex $_log_window_size/;\n use strict;\n use warnings;\n my ($ra_invalid, $can_do_switch, %ignored_err, $RA);\n@@ -5334,6 +5350,17 @@ sub get_dir_globbed {\n \t@finalents;\n }\n\n+# return value: 0 -- don't ignore, 1 -- ignore\n+sub is_ref_ignored {\n+\tmy ($g, $p) = @_;\n+\tmy $refname = $g->{ref}->full_path($p);\n+\treturn 1 if defined($g->{ignore_refs_regex}) &&\n+\t            $refname =~ m!$g->{ignore_refs_regex}!;\n+\treturn 0 unless defined($_ignore_refs_regex);\n+\treturn 1 if $refname =~ m!$_ignore_refs_regex!o;\n+\treturn 0;\n+}\n+\n sub match_globs {\n \tmy ($self, $exists, $paths, $globs, $r) = @_;\n\n@@ -5370,6 +5397,7 @@ sub match_globs {\n \t\t\tnext unless /$g->{path}->{regex}/;\n \t\t\tmy $p = $1;\n \t\t\tmy $pathname = $g->{path}->full_path($p);\n+\t\t\tnext if is_ref_ignored($g, $p);\n \t\t\tnext if $exists->{$pathname};\n \t\t\tnext if ($self->check_path($pathname, $r) !=\n \t\t\t         $SVN::Node::dir);\n-- \n1.7.4.1\n"},{"id":"177202","messageId":"7vvcs0s7xa.fsf@alter.siamese.dyndns.org","threadId":"28618","inReplyTo":"CAN4ruPiSgY+LPdDgS021WQyoHMuNrJDzrqMuCt9G5qfZ=XtjoQ@mail.gmail.com","subject":"Re: [PATCH] git-svn: Allow certain refs to be ignored","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-10-07T23:23:13Z","receivedAt":"2011-10-07T23:23:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Asking Eric to comment when he has time to do so.\n\nI find these pattern matches that are not anchored on either side \nsomewhat disturbing (e.g. --ignore-refs=master would ignore master2)\nbut ignore-paths codepath seems to follow the same pattern, so perhaps it\nis in line with what git-svn users want. I dunno.\n\nMichael Olson <mwolson@gnu.org> writes:\n\n> Implement a new --ignore-refs option which specifies a regex of refs\n> to ignore while importing svn history.\n>\n> This is a useful supplement to the --ignore-paths option, as that\n> option only operates on the contents of branches and tags, not the\n> branches and tags themselves.\n>\n> Signed-off-by: Michael Olson <mwolson@gnu.org>\n> ---\n> Re-sent by request of Piotr Krukowiecki.  This is against v1.7.4.1,\n> and I've been using it stably for a while.\n>\n>  git-svn.perl |   38 +++++++++++++++++++++++++++++++++-----\n>  1 files changed, 33 insertions(+), 5 deletions(-)\n>\n> diff --git a/git-svn.perl b/git-svn.perl\n> index 177dd25..541fa2d 100755\n> --- a/git-svn.perl\n> +++ b/git-svn.perl\n> @@ -90,7 +90,8 @@ $_q ||= 0;\n>  my %remote_opts = ( 'username=s' => \\$Git::SVN::Prompt::_username,\n>                      'config-dir=s' => \\$Git::SVN::Ra::config_dir,\n>                      'no-auth-cache' => \\$Git::SVN::Prompt::_no_auth_cache,\n> -                    'ignore-paths=s' => \\$SVN::Git::Fetcher::_ignore_regex );\n> +                    'ignore-paths=s' => \\$SVN::Git::Fetcher::_ignore_regex,\n> +                    'ignore-refs=s' => \\$Git::SVN::Ra::_ignore_refs_regex );\n>  my %fc_opts = ( 'follow-parent|follow!' => \\$Git::SVN::_follow_parent,\n>  \t\t'authors-file|A=s' => \\$_authors,\n>  \t\t'authors-prog=s' => \\$_authors_prog,\n> @@ -380,9 +381,12 @@ sub do_git_init_db {\n>  \t\tcommand_noisy('config', \"$pfx.$i\", $icv{$i});\n>  \t\t$set = $i;\n>  \t}\n> -\tmy $ignore_regex = \\$SVN::Git::Fetcher::_ignore_regex;\n> -\tcommand_noisy('config', \"$pfx.ignore-paths\", $$ignore_regex)\n> -\t\tif defined $$ignore_regex;\n> +\tmy $ignore_paths_regex = \\$SVN::Git::Fetcher::_ignore_regex;\n> +\tcommand_noisy('config', \"$pfx.ignore-paths\", $$ignore_paths_regex)\n> +\t\tif defined $$ignore_paths_regex;\n> +\tmy $ignore_refs_regex = \\$Git::SVN::Ra::_ignore_refs_regex;\n> +\tcommand_noisy('config', \"$pfx.ignore-refs\", $$ignore_refs_regex)\n> +\t\tif defined $$ignore_refs_regex;\n>  }\n>\n>  sub init_subdir {\n> @@ -1831,6 +1835,8 @@ sub read_all_remotes {\n>  \t\t\t$r->{$1}->{svm} = {};\n>  \t\t} elsif (m!^(.+)\\.url=\\s*(.*)\\s*$!) {\n>  \t\t\t$r->{$1}->{url} = $2;\n> +\t\t} elsif (m!^(.+)\\.ignore-refs=\\s*(.*)\\s*$!) {\n> +\t\t\t$r->{$1}->{ignore_refs_regex} = $2;\n>  \t\t} elsif (m!^(.+)\\.(branches|tags)=$svn_refspec$!) {\n>  \t\t\tmy ($remote, $t, $local_ref, $remote_ref) =\n>  \t\t\t                                     ($1, $2, $3, $4);\n> @@ -1867,6 +1873,16 @@ sub read_all_remotes {\n>  \t\t}\n>  \t} keys %$r;\n>\n> +\tforeach my $remote (keys %$r) {\n> +\t\tforeach ( grep { defined $_ }\n> +\t\t\t  map { $r->{$remote}->{$_} } qw(branches tags) ) {\n> +\t\t\tforeach my $rs ( @$_ ) {\n> +\t\t\t\t$rs->{ignore_refs_regex} =\n> +\t\t\t\t    $r->{$remote}->{ignore_refs_regex};\n> +\t\t\t}\n> +\t\t}\n> +\t}\n> +\n>  \t$r;\n>  }\n>\n> @@ -4876,7 +4892,7 @@ sub apply_diff {\n>  }\n>\n>  package Git::SVN::Ra;\n> -use vars qw/@ISA $config_dir $_log_window_size/;\n> +use vars qw/@ISA $config_dir $_ignore_refs_regex $_log_window_size/;\n>  use strict;\n>  use warnings;\n>  my ($ra_invalid, $can_do_switch, %ignored_err, $RA);\n> @@ -5334,6 +5350,17 @@ sub get_dir_globbed {\n>  \t@finalents;\n>  }\n>\n> +# return value: 0 -- don't ignore, 1 -- ignore\n> +sub is_ref_ignored {\n> +\tmy ($g, $p) = @_;\n> +\tmy $refname = $g->{ref}->full_path($p);\n> +\treturn 1 if defined($g->{ignore_refs_regex}) &&\n> +\t            $refname =~ m!$g->{ignore_refs_regex}!;\n> +\treturn 0 unless defined($_ignore_refs_regex);\n> +\treturn 1 if $refname =~ m!$_ignore_refs_regex!o;\n> +\treturn 0;\n> +}\n> +\n>  sub match_globs {\n>  \tmy ($self, $exists, $paths, $globs, $r) = @_;\n>\n> @@ -5370,6 +5397,7 @@ sub match_globs {\n>  \t\t\tnext unless /$g->{path}->{regex}/;\n>  \t\t\tmy $p = $1;\n>  \t\t\tmy $pathname = $g->{path}->full_path($p);\n> +\t\t\tnext if is_ref_ignored($g, $p);\n>  \t\t\tnext if $exists->{$pathname};\n>  \t\t\tnext if ($self->check_path($pathname, $r) !=\n>  \t\t\t         $SVN::Node::dir);\n"},{"id":"177203","messageId":"CAN4ruPgRed=dK8PuHCvKRBVvpSK5iDYmprZ1p_Yu8Kqyvu8sfg@mail.gmail.com","threadId":"28618","inReplyTo":"7vvcs0s7xa.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] git-svn: Allow certain refs to be ignored","fromName":"Michael Olson","fromEmail":"mwolson@gnu.org","sentAt":"2011-10-07T23:33:27Z","receivedAt":"2011-10-07T23:33:27Z","isPatch":true,"sender":{"key":"mwolson@gnu.org","avatar":"https://avatars.githubusercontent.com/u/1510193?v=4"},"body":"On Fri, Oct 7, 2011 at 4:23 PM, Junio C Hamano <gitster@pobox.com> wrote:\n> Asking Eric to comment when he has time to do so.\n>\n> I find these pattern matches that are not anchored on either side\n> somewhat disturbing (e.g. --ignore-refs=master would ignore master2)\n> but ignore-paths codepath seems to follow the same pattern, so perhaps it\n> is in line with what git-svn users want. I dunno.\n\nMy own personal use of this takes a list of patterns, concatenates\nthem into one giant pattern, adds '^', '$', and writes it out to\n.gitconfig.  So I don't really have a preference, other than to make\nboth options consistent.\n\n-- \nMichael Olson  |  http://mwolson.org/\n"},{"id":"177337","messageId":"20111010225838.GB3828@dcvr.yhbt.net","threadId":"28618","inReplyTo":"7vvcs0s7xa.fsf@alter.siamese.dyndns.org","subject":"Re: [PATCH] git-svn: Allow certain refs to be ignored","fromName":"Eric Wong","fromEmail":"normalperson@yhbt.net","sentAt":"2011-10-10T22:58:38Z","receivedAt":"2011-10-10T22:58:38Z","isPatch":true,"sender":{"key":"e@80x24.org","avatar":null},"body":"Junio C Hamano <gitster@pobox.com> wrote:\n> Asking Eric to comment when he has time to do so.\n> \n> I find these pattern matches that are not anchored on either side \n> somewhat disturbing (e.g. --ignore-refs=master would ignore master2)\n> but ignore-paths codepath seems to follow the same pattern, so perhaps it\n> is in line with what git-svn users want. I dunno.\n\nAs stated last year, I remember wanting globs instead of regexps, but\nwe already made the regexp mistake with ignore-paths, too :(\n\nI don't think it's horrible with regexps, and if git-svn users find it\nuseful, it's fine by me.\n\n> Michael Olson <mwolson@gnu.org> writes:\n> > Re-sent by request of Piotr Krukowiecki.  This is against v1.7.4.1,\n> > and I've been using it stably for a while.\n\nMichael: can you please rebase against latest and resend?  Thanks.\n"},{"id":"177630","messageId":"CAA01Csq9jR_C9t4PzsnpJo7wt=M_wnJVXZG2fAmrvvOgPb1SBA@mail.gmail.com","threadId":"28618","inReplyTo":"20111010225838.GB3828@dcvr.yhbt.net","subject":"Re: [PATCH] git-svn: Allow certain refs to be ignored","fromName":"Piotr Krukowiecki","fromEmail":"piotr.krukowiecki@gmail.com","sentAt":"2011-10-14T09:24:57Z","receivedAt":"2011-10-14T09:24:57Z","isPatch":true,"sender":{"key":"piotr.krukowiecki@gmail.com","avatar":"https://avatars.githubusercontent.com/u/3259959?v=4"},"body":"On Tue, Oct 11, 2011 at 12:58 AM, Eric Wong <normalperson@yhbt.net> wrote:\n> Junio C Hamano <gitster@pobox.com> wrote:\n>> Asking Eric to comment when he has time to do so.\n>>\n>> I find these pattern matches that are not anchored on either side\n>> somewhat disturbing (e.g. --ignore-refs=master would ignore master2)\n>> but ignore-paths codepath seems to follow the same pattern, so perhaps it\n>> is in line with what git-svn users want. I dunno.\n>\n> As stated last year, I remember wanting globs instead of regexps, but\n> we already made the regexp mistake with ignore-paths, too :(\n>\n> I don't think it's horrible with regexps, and if git-svn users find it\n> useful, it's fine by me.\n\nIn my case globs would be too limited. I'm using negative look-ahead\nassertions to match only branches/tags for projects that are\ninteresting to me. With globs it's not possible AFAIK (I would have to\nspecify all ignored patterns by hand which would work only for known\npatterns):\n  ignore-paths = ^path/branches/(?!proj1|proj2)\n\n\n>> Michael Olson <mwolson@gnu.org> writes:\n>> > Re-sent by request of Piotr Krukowiecki.  This is against v1.7.4.1,\n>> > and I've been using it stably for a while.\n>\n> Michael: can you please rebase against latest and resend?  Thanks.\n\nI've got following error while using the patch. I don't know it also\nhappens without the patch...\n\n\tM\t...\nr216099 = 4d16d4890915f4c02ba541956957a4e4b4bed400\n(refs/remotes/proj2/proj2-branch9)\nAuto packing the repository for optimum performance. You may also\nrun \"git gc\" manually. See \"git help gc\" for more information.\nCounting objects: 10284, done.\nCompressing objects: 100% (7695/7695), done.\nWriting objects: 100% (10284/10284), done.\nTotal 10284 (delta 5077), reused 0 (delta 0)\nfatal: refs/remotes/proj1/trunk: not a valid SHA1\nupdate-ref refs/heads/master refs/remotes/proj1/trunk: command\nreturned error: 128\n\n$ git log -1\nfatal: bad default revision 'HEAD'\n$ git log -1 trunk\nfatal: ambiguous argument 'trunk': unknown revision or path not in the\nworking tree.\nUse '--' to separate paths from revisions\n$ ls .git/refs/remotes/\nproj2\n\nThe commands I've used:\n\ngit svn init --prefix=proj2/ -Rproj2 -s\n--ignore-paths='^proj2/branches/(?!proj1|proj2)|^proj2/tags/(?!proj1|proj2)|^proj2/(?!trunk|branches|tags)'\n--ignore-refs='^refs/remotes/proj2/(?!proj1|proj2|trunk|tag)|^refs/remotes/proj2/tags/(?!proj1|proj2)'\nhttp://url/svn/repos/proj2\ngit svn init --prefix=proj1/ -Rproj1 -s\n--ignore-paths='^proj1/branches/(?!proj1|proj2)|^proj1/tags/(?!proj1|proj2)|^proj2/(?!trunk|branches|tags)'\n--ignore-refs='^refs/remotes/proj1/(?!proj1|proj2|trunk|tag)|^refs/remotes/proj1/tags/(?!proj1|proj2)'\nhttp://url/svn/repos/proj1\ngit svn fetch proj2 && git svn fetch proj1\n\nThe error happened while fetching proj2. I wonder why it tried to set\nmaster to proj1?\nIs my command ok? I want to have both proj1 and proj2 under one git repository.\n\n\n-- \nPiotr Krukowiecki\n"}]}