[PATCH v2 00/12] vcs-svn: incremental import

Hi again,

Jonathan Nieder wrote:

Here's a reroll. Aside from the aspects already mentioned (which avoid a dependency on the mostly orthogonal topic of support for text deltas), the original patch #10 has been split into smaller, more easily digestible pieces.

Most of the credit for this incarnation of the series belongs to David, who heroically streamlined it and untangled it from other topics.

Patch 1 changes the mark numbers for blobs to be ridiculously high, to make room for memorable commit marks (:1 for r1, :2 for r2, etc). Patch 2 brings those commit marks into existence, as mentioned before.

Patches 3-5 simplify the repo-tree API somewhat. They are somewhat minimal; patches on top of this series offering further simplification would be very welcome.

Patch 6 is a bit sneaky.  We want svn-fe's output to change from

	<import blob>
	<import blob>
	...
	<import blob>
	<import commit, using blobs>

to

	<commit header>
	M 100644 inline one/path
	<import blob>
	M 100644 inline another/path
	...
	<commit footer (progress update)>

since the latter allows svn-fe to maintain much less state. But that's a big change, so patch 6 introduces a stepping stone on the way there:

	<comment that will become commit header>
	<import blob>
	...
	<import commit; this will become the commit footer>

That paves the way for patches 7-11, which teach svn-fe to rely on the fast-import backend for information about previously imported blobs, at long last.

The visible effects should be:

 - svn-fe _requires_ a backchannel from the fast-import
   backend now.  You can't do

	svn-fe <dump >stream &&
	fast-import <stream

   in two steps any more.

 - Given one dump that picks up where another left off, svn-fe
   can continue the import.  Use

	git fast-import --relative-marks \
		--import-marks-if-exists=svn-revs \
		--export-marks=svn-revs \
		--cat-blob-fd=3 3>backchannel

   for both imports.

I'm not happy about the loss of usability but I'm happy about the gain in functionality. A good next step might be to build a simple remote helper to make this comfortable to use.

Thoughts? Improvements? Complaints? Despite the deficiencies just mentioned I'm tempted to push this out soon. Feedback in either direction would be welcome.

David Barr (3):
  vcs-svn: set up channel to read fast-import cat-blob response
  vcs-svn: quote paths correctly for ls command
  vcs-svn: use mark from previous import for parent commit

Jonathan Nieder (9):
  vcs-svn: use higher mark numbers for blobs
  vcs-svn: save marks for imported commits
  vcs-svn: introduce repo_read_path to check the content at a path
  vcs-svn: handle_node: use repo_read_path
  vcs-svn: simplify repo_modify_path and repo_copy
  vcs-svn: add a comment before each commit
  vcs-svn: allow input errors to be detected promptly
  vcs-svn: eliminate repo_tree structure
  vcs-svn: handle filenames with dq correctly

 contrib/svn-fe/svn-fe.txt |    6 +-
 t/t9010-svn-fe.sh         |  217 +++++++++++++++++++------
 vcs-svn/fast_export.c     |  145 +++++++++++++++--
 vcs-svn/fast_export.h     |   39 +++--
 vcs-svn/line_buffer.c     |    5 +
 vcs-svn/line_buffer.h     |    1 +
 vcs-svn/repo_tree.c       |  386 ++++++++-------------------------------------
 vcs-svn/repo_tree.h       |    5 +-
 vcs-svn/string_pool.c     |   13 ++-
 vcs-svn/string_pool.h     |    3 +-
 vcs-svn/svndump.c         |  106 +++++++++----
 11 files changed, 490 insertions(+), 436 deletions(-)
 rewrite vcs-svn/fast_export.h (75%)
 rewrite vcs-svn/repo_tree.c (96%)