Re: GSOC Proposal draft: git-remote-svn

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Apr 11, 2012, 19:53 UTC
Message-ID: <20120411195351.GG4248@burratino>
In-Reply-To: <2866164.rI5svgrW1x@flomedio>

Florian Achleitner wrote:

> If the remote-svn-alpha script is really all that needs to be done, you're 
> right. It just pipes through svn-fe. I thought svn-fe could only import an svn 
> repo initially, and there would be some difference between importing the whole 
> history and fetching new revisions later, (?).

Yes, Dmitry's script (not the first version, but a later one) supports incremental imports without trouble if I remember correctly.

[...]

> Listing patches and planing all details in the submitted proposal would 
> require me to know what I do and how I will do it all before last Friday! As 
> I'm not yet an expert on this topic, I don't know how I could have known all 
> details a-priori.

Oh, I didn't mean you would need to do that alone. :) Dmitry, David, Ram, Sverre, and I should be able to answer any questions you have about how git, vcs-svn, svnrdump, and the transport-helper currently work in the importer.

I've marked the proposal editable to allow details to be filled in.

[...]

> I planned to implement a remote-helper using the existing interface 
> specification to communicate over pipes with git's transport-helper. 
> Instead of invoking svn-fe as a subprocess, I want to call vcs-svn/ functions 
> directly from the remote-helper and place new functions in this directory (?).

Ah, this is a good place to start. In my diagram I lumped everything under vcs-svn/ together as svn-fe for convenience, but in fact the vcs-svn lib is made up of multiple components:

	caller
	 .
	 :
	 |
	public interface (svndump_init, svndump_read, etc)
	 |
	 |
	 |
	dump file parser (svndump_read body)
	 |
	 |
	 |
	fast-export interface (fast_export_*, repo_*) --------- svndiff0 parser
	 |
	 :
	 .
	git fast-import

Each component has a narrow interface. For each action in the dump, svndump_read() calls some appropriate function from the fast-export interface to bring about the corresponding change on the git side. Details of svndump syntax and the state needed to parse it are isolated in svndump.c and details of fast-import syntax are in fast-export.c and repo_tree.c.

(The structure used to be more complicated when the repo_* functions had to keep track of the repository state instead of relying on fast-import for that.)

Where would the branch mapping go? What kind of state needs to be maintained as it occurs? What steps would I follow to imitate the code and work out a branch mapping on paper? How do I invoke the code if I want to try it out (i.e., what functions form the public interface needed to support branch mapping)?

I don't expect you to have answers to all these questions already; I understand that getting used to what's already there and trying out ideas will take time. However, I do think we have a much better chance of this going well if there are answers to these questions by the time the coding period starts.

[...]

> Additionally the remote-helper will read a configuration file containing 
> additional information about branch-mapping, this should be closely related to 
> Andrew's SBL.

That sounds reasonable to me. I am somewhat unconvinced (but convinceable) about the need to use a configuration scheme that handles all the edge cases right away. Shouldn't it be enough to tell the importer the following?

 - the path to the repository (from which it can deduce $SVNROOT
   and the path within there to the subproject of interest)

 - a single bit of information on top of that: "this repository uses
   the standard layout"

Once that works, the tools could easily be tweaked to respect a configuration file that describes more complex situations, and as a bonus the SBL tools for making sense of those situations would have time to become more mature in the meantime.

Thanks for some useful clarifications. Jonathan