fast-import tweaks for remote helpers (Re: Status of the svn remote helper project (Dec 2010, #1))

From: Jonathan Nieder <jrnieder@gmail.com>
Date: Dec 12, 2010, 06:14 UTC
Message-ID: <20101212061437.GA17185@burratino>
In-Reply-To: <4CFFCDCD.9060602@dbservice.com>

Tomas Carnecky wrote:

> I simplified the code and the requirements on fast-import are much
> lighter now. All I need is a way to tell fast-import to stop writing
> refs and after each commit write its sha1 to stdout.

That's good to hear. What should be the syntax for asking fast-import not to write to a ref? Something like this?

	commit
	mark :1
	committer c o mitter <committer@example.com> now
	data <<END
	...

Writing the sha1 as each commit is written: how early does the frontend need access to the sha1? Would a facility to report marks back to the frontend at the end of the stream take care of it?

Based on [1] I guess the main need is some way for fast-import to tell the transport machinery what refs the transport machinery should update (or at least ought to report as updated). A hackish way might be to make the remote helper send "progress" commands with that information.

Show 5 quoted lines

> It's possible to
> to modify fast-import.c with a small patch to make it behave like
> that. However, I haven't followed the svn remote helper that much
> lately so I don't know whether one of the other patches already
> modifies fast-import in this way.

No, the patches have mostly been adding commands that send information back to the frontend.

 cat-blob (<dataref> | <mark>):
	Sends back an old blob along with its length (in
	cat-file --batch format).  svn-fe uses this to acquire
	the preimage when applying deltas.

 ls <quoted-path>:
	Sends back information about the current state of a path
	in the commit being prepared (as a single line in ls-tree
	format).  svn-fe uses this to move around files and to find
	a <dataref> to use with cat-blob when applying deltas.

 ls (<dataref> | <mark>) <path>:
	Sends back information about a path in a previous revision
	(tag, commit, or tree), in ls-tree format.

 M 040000 (<dataref> | <mark>) <path>:
	Like "M 100644 <dataref> <path>", replaces an entry in the
	active commit with content of the frontend's choice.  This
	gets used to copy in old directories.

> From the beginning my code was meant to be just an example how the
> interaction between git and the svn remote helper could look like.

It makes a nice demo, too. :)

> For example I save the svn rev <-> sha1 mapping in notes, which is
> appears to work well. I'll take a look if I'll be able to use the
> svn-fe in my script.

svn-fe needs a fast mapping svn rev -> sha1; it currently uses a marks file for that. (In the back of my mind, I have the idea of using a file that allows O(1) access, perhaps of the form

	<commit name for rev 1> NL
	<commit name for rev 2> NL
	...

but as Ram has noted, keeping the whole table in memory is pretty cheap already.) A remote helper needs a fast mapping sha1 -> svn rev, and imho notes are ideal for that[2].

The way I imagine it, the authoritative mapping is in notes and the reverse mapping (e.g. in a marks file) is rebuilt when needed.

[1] remote-helper branch at git://github.com/wereHamster/git.git [2] Why? When a project switches from one svn server to another, revision numbers tend to change, so revision numbers are not permanent enough to belong in the commit message imho. (If only git-notes had existed when git svn was written...)