threads / rfc / 23540

RFC patch, 2 partsgit-svn: Allow certain refs to be ignored

Subject: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

## tl;dr

7 messages between Apr 20, 2010 and Apr 21, 2010. Diffs are folded; open one to read it.

replies: 6people: 3as markdown or json

Michael Olson· Apr 20, 2010, 21:30 UTC · lore

This change allows certain refs to be ignored while importing svn history. This is useful for:

 - Performing overlapping transforms from svn's branch namespace into
   git's namespace.  The overlapping refs can be ignored.  It might
   potentially be a better idea in this case to make git-svn assign
   precedence according to the order in which the branch patterns are
   specified, perhaps with an option to trigger that behavior (which
   is out of scope for this patch series).
 - Avoiding some refs that represent bad svn operations which cause
   git-svn to take a very long time.  Example: copying one module
   accidentally into another module's branch namespace.

A new config directive called "ignore-refs" implements this feature by means of a regex of refs to ignore. I haven't written the necessary git-svn documentation updates for it yet.

In addition, I ran across a problem where git-svn would die if a parent ref did not exist. This might possibly have been exposed as a result of the first patch. I threw an eval around the offending code, and that seems to work, though some review would be appreciated.

Here is an example ~/.git/config file which works with these changes. The example upstream svn repository has branches in tags in 2 different namespaces. Originally all tags/branches were directly under /root/mod/branches and /root/mod/tags. Later on, they created new branches and tags in subdirectories such as /root/mod/branches/myorg/bugs/BUGID. This config file forces old-style branches and tags into the "old/" namespace in git, and puts new-style branches and tags into (for example) "myorg/bugs/BUGID". The "ignore-refs" directive prevents the first level of the new namespace ("myorg", in particular) from being replicated underneath "old/" in git.

[core]
       repositoryformatversion = 0
       filemode = true
       bare = false
       logallrefupdates = true
[svn-remote "svn"]
       url = https://svn.my.org/svn/root
       fetch = mod/trunk:refs/remotes/trunk
       branches = mod/branches/*:refs/remotes/old/*
       tags = mod/tags/*:refs/remotes/tags/old/*
       branches = mod/branches/myorg/bugs/*:refs/remotes/myorg/bugs/*
       tags = mod/tags/myorg/bugs/*:refs/remotes/tags/myorg/bugs/*
       branches = mod/branches/myorg/projects/*:refs/remotes/myorg/projects/*
       tags = mod/tags/myorg/projects/*:refs/remotes/tags/myorg/projects/*
       branches = mod/branches/myorg/releases/*:refs/remotes/myorg/releases/*
       tags = mod/tags/myorg/releases/*:refs/remotes/tags/myorg/releases/*
       ignore-refs = ^refs/remotes/(tags/)?old/myorg
Michael Olson (2):
  git-svn: Allow certain refs to be ignored
  git-svn: Don't allow missing commit parent to stop git-svn
 git-svn.perl |   40 ++++++++++++++++++++++++++++++++++------
 1 files changed, 34 insertions(+), 6 deletions(-)
Junio C Hamano· Apr 20, 2010, 23:58 UTC · re: Michael Olson · lore

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

Michael Olson <mwolson@gnu.org> writes:
> [svn-remote "svn"]
>        url = https://svn.my.org/svn/root
> ...
>        ignore-refs = ^refs/remotes/(tags/)?old/myorg

Traditionally configuration variable names are spelled camelCase without dashes. You probably would want to be consistent.

Also "refs" and any pathname-like things are traditionally matched using globs and not regexes. It is Ok to deviate if you have a strong reason to (and I suspect it would make it easier to write "exclude" patterns like the above example to allow a regex here), but that needs to be prominently documented (e.g. "Unlike any other ref-matching configuration variable, this alone uses regex, not glob") to avoid end user confusion.

Eric Wong· Apr 21, 2010, 07:15 UTC · re: Junio C Hamano · lore

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

Junio C Hamano <gitster@pobox.com> wrote:
Show 9 quoted lines
> Michael Olson <mwolson@gnu.org> writes:
> 
> > [svn-remote "svn"]
> >        url = https://svn.my.org/svn/root
> > ...
> >        ignore-refs = ^refs/remotes/(tags/)?old/myorg
> 
> Traditionally configuration variable names are spelled camelCase without
> dashes.  You probably would want to be consistent.

Configuration variables should definitely be camelCase in examples/documentation (and I even dislike camelCase). No dashes or underscores here.

Show 6 quoted lines
> Also "refs" and any pathname-like things are traditionally matched using
> globs and not regexes.  It is Ok to deviate if you have a strong reason to
> (and I suspect it would make it easier to write "exclude" patterns like
> the above example to allow a regex here), but that needs to be prominently
> documented (e.g. "Unlike any other ref-matching configuration variable,
> this alone uses regex, not glob") to avoid end user confusion.
I favor globs for more consistent/natural for path matching.

Another thing to keep in mind is that these would be Perl regular expressions exposed to a user interface. If git-svn were ever be reimplemented in something other than Perl, PCRE (or similar) would be required to interpret them consistently.

-- 
Eric Wong
Michael Olson· Apr 21, 2010, 16:31 UTC · re: Junio C Hamano · lore

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

On Tue, Apr 20, 2010 at 4:58 PM, Junio C Hamano <gitster@pobox.com> wrote:
Show 9 quoted lines
> Michael Olson <mwolson@gnu.org> writes:
>
>> [svn-remote "svn"]
>>        url = https://svn.my.org/svn/root
>> ...
>>        ignore-refs = ^refs/remotes/(tags/)?old/myorg
>
> Traditionally configuration variable names are spelled camelCase without
> dashes.  You probably would want to be consistent.

I was looking at the existing syntax for "ignore-paths", which uses dashes. Should that be changed as well?

Show 6 quoted lines
> Also "refs" and any pathname-like things are traditionally matched using
> globs and not regexes.  It is Ok to deviate if you have a strong reason to
> (and I suspect it would make it easier to write "exclude" patterns like
> the above example to allow a regex here), but that needs to be prominently
> documented (e.g. "Unlike any other ref-matching configuration variable,
> this alone uses regex, not glob") to avoid end user confusion.

ignore-paths also uses a regex. I'm concerned that using globs will not be expressive enough to represent a regex like (a combination of the 2 use cases I posted initially):

^refs/remotes/((tags/)?old/myorg|old/bad_branch|old/unlabeled[^/]*|tags/(old/bad_tag|releases/another_tag))$
-- 
Michael Olson  |  http://mwolson.org/
Eric Wong· Apr 21, 2010, 16:54 UTC · re: Michael Olson · lore

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

Michael Olson <mwolson@gnu.org> wrote:
Show 13 quoted lines
> On Tue, Apr 20, 2010 at 4:58 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Michael Olson <mwolson@gnu.org> writes:
> >
> >> [svn-remote "svn"]
> >>        url = https://svn.my.org/svn/root
> >> ...
> >>        ignore-refs = ^refs/remotes/(tags/)?old/myorg
> >
> > Traditionally configuration variable names are spelled camelCase without
> > dashes.  You probably would want to be consistent.
> 
> I was looking at the existing syntax for "ignore-paths", which uses
> dashes.  Should that be changed as well?
Yes, I missed that the first time around :x
I suppose that should be changed to work both ways.
Show 10 quoted lines
> > Also "refs" and any pathname-like things are traditionally matched using
> > globs and not regexes.  It is Ok to deviate if you have a strong reason to
> > (and I suspect it would make it easier to write "exclude" patterns like
> > the above example to allow a regex here), but that needs to be prominently
> > documented (e.g. "Unlike any other ref-matching configuration variable,
> > this alone uses regex, not glob") to avoid end user confusion.
> 
> ignore-paths also uses a regex.  I'm concerned that using globs will
> not be expressive enough to represent a regex like (a combination of
> the 2 use cases I posted initially):
I also missed that (I never used that feature :x)
> ^refs/remotes/((tags/)?old/myorg|old/bad_branch|old/unlabeled[^/]*|tags/(old/bad_tag|releases/another_tag))$

I wonder if both would be better as a series of globs would be easier to read (maybe more verbose to write):

	ignoreRefs = refs/remotes/tags/old/myorg
	ignoreRefs = refs/remotes/old/myorg
	ignoreRefs = refs/remotes/old/unlabeled/*

But I suppose the regexp route is fine. I'll ack and push them out with Sam's Ack on 2/2

-- 
Eric Wong
Michael Olson· Apr 21, 2010, 17:02 UTC · re: Eric Wong · lore

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

On Wed, Apr 21, 2010 at 9:54 AM, Eric Wong <normalperson@yhbt.net> wrote:
Show 9 quoted lines
> Michael Olson <mwolson@gnu.org> wrote:
>> ^refs/remotes/((tags/)?old/myorg|old/bad_branch|old/unlabeled[^/]*|tags/(old/bad_tag|releases/another_tag))$
>
> I wonder if both would be better as a series of globs would be
> easier to read (maybe more verbose to write):
>
>        ignoreRefs = refs/remotes/tags/old/myorg
>        ignoreRefs = refs/remotes/old/myorg
>        ignoreRefs = refs/remotes/old/unlabeled/*
That does look cleaner, and kind of reminiscent of .gitignore.
-- 
Michael Olson  |  http://mwolson.org/
Eric Wong· Apr 21, 2010, 07:18 UTC · re: Michael Olson · lore

Re: [PATCH/RFC 0/2] git-svn: Allow certain refs to be ignored

Michael Olson <mwolson@gnu.org> wrote:
Show 13 quoted lines
> This change allows certain refs to be ignored while importing svn
> history.  This is useful for:
> 
>  - Performing overlapping transforms from svn's branch namespace into
>    git's namespace.  The overlapping refs can be ignored.  It might
>    potentially be a better idea in this case to make git-svn assign
>    precedence according to the order in which the branch patterns are
>    specified, perhaps with an option to trigger that behavior (which
>    is out of scope for this patch series).
> 
>  - Avoiding some refs that represent bad svn operations which cause
>    git-svn to take a very long time.  Example: copying one module
>    accidentally into another module's branch namespace.
Hi Michael, these are definitely good things to have.  Thanks!
> A new config directive called "ignore-refs" implements this feature by
> means of a regex of refs to ignore.  I haven't written the necessary
> git-svn documentation updates for it yet.
See Junio's and my other email.
> In addition, I ran across a problem where git-svn would die if a
> parent ref did not exist.  This might possibly have been exposed as a
> result of the first patch.  I threw an eval around the offending code,
> and that seems to work, though some review would be appreciated.

I'm cc-ing Sam for this one. It looks good to me given the first patch would cause more refs to not exist.

-- 
Eric Wong

← back to recent threads