threads / bug / 13019

Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

Subject: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

## tl;dr

6 messages between Apr 8, 2008 and Apr 9, 2008.

replies: 5people: 2as markdown or json

Thomas Leonard· Apr 8, 2008, 15:48 UTC · lore

Hi.

The git-svn Ubuntu packager asked me to forward this here.

Summary:

When converting a subversion repository to GIT using git-svn, the tags do not have the right parent. Each tag should be identical to a trunk revision (which it was copied from), but because git-svn uses an earlier revision as the parent it appears that the same work was duplicated on two branches.

Example:

git-svn clone https://zero-install.svn.sourceforge.net/svnroot/zero-install -T trunk/0publish -r1890:2072 -b releases/0publish

The git branch comes from r1894, yet the svn log shows that in includes files from r2070:

$ svn log https://zero-install.svn.sourceforge.net/svnroot/zero-install
-r2071 -v
r2071 | tal197 | 2007-11-10 19:40:45 +0000 (Sat, 10 Nov 2007) | 1 line
Changed paths:
   A /releases/0publish/0publish-0.12 (from /trunk/0publish:1968)
   R /releases/0publish/0publish-0.12/0publish (from
/trunk/0publish/0publish:2070)
   R /releases/0publish/0publish-0.12/0publish.xml (from
/trunk/0publish/0publish.xml:2070)
   R /releases/0publish/0publish-0.12/release.py (from
/trunk/0publish/release.py:2069)

Released 0.12

https://bugs.launchpad.net/ubuntu/+source/git-core/+bug/163341

Thanks,

---------- Forwarded message ----------
From: Emil Sit <sit@amsterdam.lcs.mit.edu>
Date: 7 Apr 2008 20:56
Subject: [Bug 163341] Re: git-svn gets wrong parent revision for tags

I know that git-svn may rely on various conventions for the use of git
 and use of svn but since I'm not really familiar with either, it sounds
 like this should best be handled by the upstream developers.  Git does
 not use a dedicated bug tracker but handles issues on the mailing list
 git@vger.kernel.org.  If you submit your patch there, you should get a
 pretty fast response and you can follow up with this bug.  In any event,
 if the bug is fixed upstream, the next version of Ubuntu will probably
 include the fix automatically.

 Can you handle the follow-up with the official git developers?  Thanks!

 --
 git-svn gets wrong parent revision for tags
 https://bugs.launchpad.net/bugs/163341
 You received this bug notification because you are a direct subscriber
 of the bug.

-- 
Dr Thomas Leonard		http://rox.sourceforge.net
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1

Björn Steinbrink· Apr 8, 2008, 17:15 UTC · re: Thomas Leonard · lore

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

On 2008.04.08 16:48:03 +0100, Thomas Leonard wrote:

Show 31 quoted lines

> Hi.
> 
> The git-svn Ubuntu packager asked me to forward this here.
> 
> Summary:
> 
> When converting a subversion repository to GIT using git-svn, the tags
> do not have the right parent. Each tag should be identical to a trunk
> revision (which it was copied from), but because git-svn uses an
> earlier revision as the parent it appears that the same work was
> duplicated on two branches.
> 
> Example:
> 
> git-svn clone https://zero-install.svn.sourceforge.net/svnroot/zero-install
> -T trunk/0publish -r1890:2072 -b releases/0publish
> 
> The git branch comes from r1894, yet the svn log shows that in
> includes files from r2070:
> 
> $ svn log https://zero-install.svn.sourceforge.net/svnroot/zero-install
> -r2071 -v
> r2071 | tal197 | 2007-11-10 19:40:45 +0000 (Sat, 10 Nov 2007) | 1 line
> Changed paths:
>    A /releases/0publish/0publish-0.12 (from /trunk/0publish:1968)
>    R /releases/0publish/0publish-0.12/0publish (from
> /trunk/0publish/0publish:2070)
>    R /releases/0publish/0publish-0.12/0publish.xml (from
> /trunk/0publish/0publish.xml:2070)
>    R /releases/0publish/0publish-0.12/release.py (from
> /trunk/0publish/release.py:2069)

Well, SVN recorded useless, broken metadata.

SVN itself believes that the branch was created from revision 1968. As that revision didn't introduce any changes to trunk/0publish, there's no commit for that revision in the git branch, so git-svn took the most recent one instead (1894).

For the other three files, SVN reports that the files were replaced by versions from another branch. There's no immediate way to tell whether those replacements make the branch equal to the more recent version of trunk. So git-svn does it the safe way and reproduces what SVN told it to reproduce: A commit that creates a branch and changes some files.

I guess sth. like this happened on the svn end: svn cp trunk/0publish releases/0publish (at rev. 1968) svn cp trunk/0publish/release.py releases/0publish (at rev. 2069) ... svn commit (whenever)

So the branch was "incrementally" created locally and SVN decided to record the "Uh, I copied this from here to there" metacrap^H^H^H^Hdata, of which git-svn made use. Hooray for rename/copy tracking.

You could maybe ask for an enhancement so that git-svn tries to figure out if the provided metadata is broken and looks up a matching revision in such a case. But I guess it's not worth the trouble and could lead to other breakage along the way...

Björn

Thomas Leonard· Apr 8, 2008, 20:43 UTC · re: Björn Steinbrink · lore

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

On 08/04/2008, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:

> On 2008.04.08 16:48:03 +0100, Thomas Leonard wrote:

[...]

Show 5 quoted lines

>  > When converting a subversion repository to GIT using git-svn, the tags
>  > do not have the right parent. Each tag should be identical to a trunk
>  > revision (which it was copied from), but because git-svn uses an
>  > earlier revision as the parent it appears that the same work was
>  > duplicated on two branches.

[...]

Show 19 quoted lines

>  > git-svn clone https://zero-install.svn.sourceforge.net/svnroot/zero-install
>  > -T trunk/0publish -r1890:2072 -b releases/0publish
>  >
>  > The git branch comes from r1894, yet the svn log shows that in
>  > includes files from r2070:
>  >
>  > $ svn log https://zero-install.svn.sourceforge.net/svnroot/zero-install
>  > -r2071 -v
>  > r2071 | tal197 | 2007-11-10 19:40:45 +0000 (Sat, 10 Nov 2007) | 1 line
>  > Changed paths:
>  >    A /releases/0publish/0publish-0.12 (from /trunk/0publish:1968)
>  >    R /releases/0publish/0publish-0.12/0publish (from
>  > /trunk/0publish/0publish:2070)
>  >    R /releases/0publish/0publish-0.12/0publish.xml (from
>  > /trunk/0publish/0publish.xml:2070)
>  >    R /releases/0publish/0publish-0.12/release.py (from
>  > /trunk/0publish/release.py:2069)
>
> Well, SVN recorded useless, broken metadata.

This is certainly true ;-)

>  SVN itself believes that the branch was created from revision 1968. As
>  that revision didn't introduce any changes to trunk/0publish, there's no
>  commit for that revision in the git branch, so git-svn took the most
>  recent one instead (1894).

Show 11 quoted lines

>  For the other three files, SVN reports that the files were replaced by
>  versions from another branch. There's no immediate way to tell whether
>  those replacements make the branch equal to the more recent version of
>  trunk. So git-svn does it the safe way and reproduces what SVN told it
>  to reproduce: A commit that creates a branch and changes some files.
>
>  I guess sth. like this happened on the svn end:
>  svn cp trunk/0publish releases/0publish (at rev. 1968)
>  svn cp trunk/0publish/release.py releases/0publish (at rev. 2069)
>  ...
>  svn commit (whenever)

The script that made the releases simply did this ("." is the working copy):

svn cp -m "Released $VERSION" . http://...

Since each file in an svn working copy has its own revision (the last time it was changed), the branch ends up with multiple source revisions, even if the working copy is fully up-to-date and has no local changes.

Example:

cd /tmp svnadmin create test-repo svn mkdir file:///tmp/test-repo/trunk -m 'Setup' svn co file:///tmp/test-repo/trunk cd trunk touch foo bar svn add foo bar svn ci -m 'Start' echo hi > bar svn ci -m 'Update' svn cp . file:///tmp/test-repo/release -m 'Release' svn log -v file:///tmp/test-repo

The log shows:

------------------------------------------------------------------------
r4 | talex | 2008-04-08 21:35:44 +0100 (Tue, 08 Apr 2008) | 1 line
Changed paths:
   A /release (from /trunk:1)
   A /release/bar (from /trunk/bar:3)
   A /release/foo (from /trunk/foo:2)

Release

So, I don't think the metadata is broken in this case. Maybe other people don't create branches like this, but it seemed like the obvious way to do it at the time. Given this behaviour of svn, perhaps it would be better to take the highest version number as the branch point?

-- 
Dr Thomas Leonard		http://rox.sourceforge.net
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1

Björn Steinbrink· Apr 9, 2008, 08:03 UTC · re: Thomas Leonard · lore

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

[Grmpf, the next time you drop from me Cc:, I'll not bother to answer...]

On 2008.04.08 21:43:03 +0100, Thomas Leonard wrote:

Show 82 quoted lines

> On 08/04/2008, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> > On 2008.04.08 16:48:03 +0100, Thomas Leonard wrote:
> [...]
> >  > When converting a subversion repository to GIT using git-svn, the tags
> >  > do not have the right parent. Each tag should be identical to a trunk
> >  > revision (which it was copied from), but because git-svn uses an
> >  > earlier revision as the parent it appears that the same work was
> >  > duplicated on two branches.
> [...]
> >  > git-svn clone https://zero-install.svn.sourceforge.net/svnroot/zero-install
> >  > -T trunk/0publish -r1890:2072 -b releases/0publish
> >  >
> >  > The git branch comes from r1894, yet the svn log shows that in
> >  > includes files from r2070:
> >  >
> >  > $ svn log https://zero-install.svn.sourceforge.net/svnroot/zero-install
> >  > -r2071 -v
> >  > r2071 | tal197 | 2007-11-10 19:40:45 +0000 (Sat, 10 Nov 2007) | 1 line
> >  > Changed paths:
> >  >    A /releases/0publish/0publish-0.12 (from /trunk/0publish:1968)
> >  >    R /releases/0publish/0publish-0.12/0publish (from
> >  > /trunk/0publish/0publish:2070)
> >  >    R /releases/0publish/0publish-0.12/0publish.xml (from
> >  > /trunk/0publish/0publish.xml:2070)
> >  >    R /releases/0publish/0publish-0.12/release.py (from
> >  > /trunk/0publish/release.py:2069)
> >
> > Well, SVN recorded useless, broken metadata.
> 
> This is certainly true ;-)
> 
> >  SVN itself believes that the branch was created from revision 1968. As
> >  that revision didn't introduce any changes to trunk/0publish, there's no
> >  commit for that revision in the git branch, so git-svn took the most
> >  recent one instead (1894).
> 
> >  For the other three files, SVN reports that the files were replaced by
> >  versions from another branch. There's no immediate way to tell whether
> >  those replacements make the branch equal to the more recent version of
> >  trunk. So git-svn does it the safe way and reproduces what SVN told it
> >  to reproduce: A commit that creates a branch and changes some files.
> >
> >  I guess sth. like this happened on the svn end:
> >  svn cp trunk/0publish releases/0publish (at rev. 1968)
> >  svn cp trunk/0publish/release.py releases/0publish (at rev. 2069)
> >  ...
> >  svn commit (whenever)
> 
> The script that made the releases simply did this ("." is the working copy):
> 
> svn cp -m "Released $VERSION" . http://...
> 
> Since each file in an svn working copy has its own revision (the last
> time it was changed), the branch ends up with multiple source
> revisions, even if the working copy is fully up-to-date and has no
> local changes.
> 
> Example:
> 
> cd /tmp
> svnadmin create test-repo
> svn mkdir file:///tmp/test-repo/trunk -m 'Setup'
> svn co file:///tmp/test-repo/trunk
> cd trunk
> touch foo bar
> svn add foo bar
> svn ci -m 'Start'
> echo hi > bar
> svn ci -m 'Update'
> svn cp . file:///tmp/test-repo/release -m 'Release'
> svn log -v file:///tmp/test-repo
> 
> The log shows:
> 
> ------------------------------------------------------------------------
> r4 | talex | 2008-04-08 21:35:44 +0100 (Tue, 08 Apr 2008) | 1 line
> Changed paths:
>    A /release (from /trunk:1)
>    A /release/bar (from /trunk/bar:3)
>    A /release/foo (from /trunk/foo:2)
> 
> Release

Change the "." to "file:///tmp/test-repo/trunk" in the svn cp command
and you get:
Changed paths:
   A /release (from /trunk:3)

Seems more useful. No idea why SVN decides to record crap when you use ".", even when your working copy is clean.

Show 5 quoted lines

> So, I don't think the metadata is broken in this case. Maybe other
> people don't create branches like this, but it seemed like the obvious
> way to do it at the time. Given this behaviour of svn, perhaps it
> would be better to take the highest version number as the branch
> point?

Uhm, and what happens then, when you actually _did_ just copy over a few files, but not all of them? Right, you get a branch to start at a later revision and see a bunch of reverts. Equally broken.

Björn

Thomas Leonard· Apr 9, 2008, 10:39 UTC · re: Björn Steinbrink · lore

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

On 09/04/2008, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:

> [Grmpf, the next time you drop from me Cc:, I'll not bother to
>  answer...]

Fixed; sorry.

>  On 2008.04.08 21:43:03 +0100, Thomas Leonard wrote:

[...]

Show 42 quoted lines

>  > The script that made the releases simply did this ("." is the working copy):
>  >
>  > svn cp -m "Released $VERSION" . http://...
>  >
>  > Since each file in an svn working copy has its own revision (the last
>  > time it was changed), the branch ends up with multiple source
>  > revisions, even if the working copy is fully up-to-date and has no
>  > local changes.
>  >
>  > Example:
>  >
>  > cd /tmp
>  > svnadmin create test-repo
>  > svn mkdir file:///tmp/test-repo/trunk -m 'Setup'
>  > svn co file:///tmp/test-repo/trunk
>  > cd trunk
>  > touch foo bar
>  > svn add foo bar
>  > svn ci -m 'Start'
>  > echo hi > bar
>  > svn ci -m 'Update'
>  > svn cp . file:///tmp/test-repo/release -m 'Release'
>  > svn log -v file:///tmp/test-repo
>  >
>  > The log shows:
>  >
>  > ------------------------------------------------------------------------
>  > r4 | talex | 2008-04-08 21:35:44 +0100 (Tue, 08 Apr 2008) | 1 line
>  > Changed paths:
>  >    A /release (from /trunk:1)
>  >    A /release/bar (from /trunk/bar:3)
>  >    A /release/foo (from /trunk/foo:2)
>  >
>  > Release
>
> Change the "." to "file:///tmp/test-repo/trunk" in the svn cp command
>  and you get:
>  Changed paths:
>    A /release (from /trunk:3)
>
>  Seems more useful. No idea why SVN decides to record crap when you use
>  ".", even when your working copy is clean.

That's good advice, but I'm wondering how to convert my existing svn repositories to GIT.

Show 9 quoted lines

>  > So, I don't think the metadata is broken in this case. Maybe other
>  > people don't create branches like this, but it seemed like the obvious
>  > way to do it at the time. Given this behaviour of svn, perhaps it
>  > would be better to take the highest version number as the branch
>  > point?
>
> Uhm, and what happens then, when you actually _did_ just copy over a
>  few files, but not all of them? Right, you get a branch to start at a
>  later revision and see a bunch of reverts. Equally broken.

Sure, if you create a situation that GIT can't represent then there's nothing much git-svn can do.

However, when there are different source revisions:

- Taking the highest revision will give correct results in the common
case, and "equally broken" results in the uncommon case.

- Taking any other revision (the current behaviour) is guaranteed to
give the wrong result. There's no way a branch that was created with a
file from revision 6 can possibly be a branch of revision 5, for
example.

Also, the current behaviour can be wildly out (e.g. revision 1894 when it should have been 2070, in the previous real example). The highest revision is unlikely to be far off.

-- 
Dr Thomas Leonard		http://rox.sourceforge.net
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1

Björn Steinbrink· Apr 9, 2008, 11:28 UTC · re: Thomas Leonard · lore

Re: Fwd: [Bug 163341] Re: git-svn gets wrong parent revision for tags

On 2008.04.09 11:39:17 +0100, Thomas Leonard wrote:

Show 23 quoted lines

> On 09/04/2008, Björn Steinbrink <B.Steinbrink@gmx.de> wrote:
> >  On 2008.04.08 21:43:03 +0100, Thomas Leonard wrote:
> >  > The log shows:
> >  >
> >  > ------------------------------------------------------------------------
> >  > r4 | talex | 2008-04-08 21:35:44 +0100 (Tue, 08 Apr 2008) | 1 line
> >  > Changed paths:
> >  >    A /release (from /trunk:1)
> >  >    A /release/bar (from /trunk/bar:3)
> >  >    A /release/foo (from /trunk/foo:2)
> >  >
> >  > Release
> >
> > Change the "." to "file:///tmp/test-repo/trunk" in the svn cp command
> >  and you get:
> >  Changed paths:
> >    A /release (from /trunk:3)
> >
> >  Seems more useful. No idea why SVN decides to record crap when you use
> >  ".", even when your working copy is clean.
> 
> That's good advice, but I'm wondering how to convert my existing svn
> repositories to GIT.

I'd probably patch git-svn locally once and revert it after the import, if your suggested change really makes sense for your repo. Another option would be to adjust the branches afterwards, but that's probably awkward in your case. Maybe a patch that allows to toggle the branch creation logic would be accepted as well, dunno.

Show 22 quoted lines

> >  > So, I don't think the metadata is broken in this case. Maybe other
> >  > people don't create branches like this, but it seemed like the obvious
> >  > way to do it at the time. Given this behaviour of svn, perhaps it
> >  > would be better to take the highest version number as the branch
> >  > point?
> >
> > Uhm, and what happens then, when you actually _did_ just copy over a
> >  few files, but not all of them? Right, you get a branch to start at a
> >  later revision and see a bunch of reverts. Equally broken.
> 
> Sure, if you create a situation that GIT can't represent then there's
> nothing much git-svn can do.
> 
> However, when there are different source revisions:
> 
> - Taking the highest revision will give correct results in the common
> case, and "equally broken" results in the uncommon case.
> 
> - Taking any other revision (the current behaviour) is guaranteed to
> give the wrong result. There's no way a branch that was created with a
> file from revision 6 can possibly be a branch of revision 5, for
> example.

Sure it can be.

svn co $repo full-repo-here cd full-repo-here svn cp trunk branches/b1 (at rev 5) echo 123 > trunk/foo # (new file) echo 456 > trunk/bar # 431 other files changed (exaggeration ;-)) svn commit trunk -m "Committing trunk only, with 433 files changed" svn cp trunk/foo branches/b1 svn commit branches/b1 -m "Committing b1."

b1 branched at rev 5, but has a single file from rev 6.

Currently, git-svn would create the branch starting at rev5, with the addition of "foo" on top.

Your suggested change would make it branch at rev6 and revert 432 files.

The fact that svn just has directory copies as branches makes all kinds of junk possible ;-)

> Also, the current behaviour can be wildly out (e.g. revision 1894 when
> it should have been 2070, in the previous real example). The highest
> revision is unlikely to be far off.

How many "interesting" revisions was it off by? 3? As the revision numbers in svn are shared throughout the whole repo, your example sounds far worse than it really is.

And for locally prepared branches (I've actually heard of people doing that, like in my example above), it can happen for your way to be way more off than the current way.

Björn