Re: [spf:guess] Re: Approaches to SVN to Git conversion (was: Re: [RFC] "Remote helper for Subversion" project)

From: Andrew Sayers <andrew-git@pileofstuff.org>
Date: Mar 7, 2012, 22:06 UTC
Message-ID: <4F57DBF0.4060101@pileofstuff.org>
In-Reply-To: <4F56A4DF.8060807@vilain.net>

It sounds like we've approached two similar problems in similar ways, so I'm curious about the differences where they exist. I've been reading this message of yours from 18 months ago alongside this thread: http://article.gmane.org/gmane.comp.version-control.git/150007 Unfortunately these comprise everything I know about Perforce.

I notice that git-p4raw stores all of its data in Postgres and provides a programmatic interface for querying it, whereas I've focussed on providing ASCII interfaces at relevant points. I can see how a DB store would help manage the amount of data you'd need to process in a big repository, but were there any other issues that drove you down this route? Did you consider a text-based interface?

On 06/03/12 23:59, Sam Vilain wrote: <snip>

Show 6 quoted lines

> What I did for the Perl Perforce conversion is make this a multi–step
> process; first, the heuristic goes through and detects branches and
> merge parents.  Then you do the actual export.  If, however, the
> heuristic gets it wrong, then you can manually override the branch
> detection for a particular revision, which invalidates all of the
> _automatic_ decisions made for later revisions the next time you run it.

Could you give an example of overriding branch/merge detection? It sounds like you're saying that if there's some problem detecting merge parents in an early revision, then all future merges are ignored by the script.

<snip>

> The manual input is extremely useful for bespoke conversions; there will
> always be warts in the history and no heuristic is perfect (even if you
> can supply your own set of expressions, a way to override it for just
> one revision is handy).

Again, would you mind providing a few examples? It sounds like you have some edge cases that could be handled by extending the branch history format, but I'd like to pin it down a bit more before discussing solutions.

<snip>

>   3. skip bad sections of history, for instance squash merging merges
> which happened over several commits (SVN and Perforce, of course,
> support insane piecemeal merging prohibited by git)

This is an excellent point I've stumbled past in my experiments without realising what I was seeing. A simple SVN example might look like this:

	svn add trunk branches
	svn add trunk/foo trunk/bar
	svn ci -m "Initial revision" # r1

	svn cp trunk branches/my_branch
	svn ci -m "Created my_branch" # r2

	# edit files in my_branch

	svn merge branches/my_branch/foo trunk/foo
	svn ci -m "Merge my_branch -> trunk (1/3)" # r11

	svn merge branches/my_branch/bar trunk/bar
	svn ci -m "Merge my_branch -> trunk (2/3)" # r12

	svn cp branches/my_branch/new_file trunk/new_file
	svn ci -m "Merge my_branch -> trunk (3/3)" # r13

This strikes me as a sensibly cautious workflow in SVN, where merge conflicts are common and changes are hard to revert. The best representation for this in the current branch history format would be something like this:

	In r1, create branch "trunk"
	In r2, create branch "branches/my_branch" from "trunk"
	In r13, merge "branches/my_branch" r13 into "trunk"

In other words, pretend r11 and r12 are just normal commits, and that r13 is a full merge. A more useful (and arguably more accurate) representation would be possible if we extended the format a bit:

	In r1, create branch "trunk"
	In r2, create branch "branches/my_branch" from "trunk"
	In r12, squash changes in "branches/my_branch"
	In r13, squash changes in "branches/my_branch"
	In r13, merge "branches/my_branch" r13 into "trunk"

Adding "squash" and "fixup" commands would let us represent the whole messy business as a single commit, which is closer to what the user was trying to say even if it's further from what they actually had to say.

	- Andrew