[RFC/PATCH 0/3] fast-import: give importers access to the object store

Hi Ram,

Ramkumar Ramachandra wrote:

Ok.

To recap, because fast-import does not write a pack index until the checkpoint, frontends cannot necessarily read back what they have written immediately. The obvious application to allowing that is as a sanity check, but more important to us is that it allows the lazy frontend to forget previous revisions even if they are required for later "file/directory copy" operations in the import. The frontend can be secure in the knowledge that the backend remembers everything now.

It works like this:

frontend:
	feature report-fd=3
	commit refs/heads/master
	... revision 1 ...

importer:
	abc7856cba76bca87a65bca76bca8bca98bca78bca76

frontend:
	commit refs/heads/master
	... revision 2 ...

importer:
	fcdafecfdacba667cd5a4da6dca5fa68ca897dc65178

...

frontend:
	commit refs/heads/master
	... revision 50 ...

	cat abc7856cba76bca87a65bca76bca8bca98bca78

importer:
	c78a67d9987da089cd89ac879dacd897acd879acd76
	commit abc7856cba76bca87a65bca76bca8bca98bca78bca76 274819
	tree <... tree name ...>
	author <... author name ...>
	...

I suspect that these patches are not in their final form. In particular, the interface is kind of klunky: to name a blob by path, you have to supply a *tree* which contains that blob as well as the pathname. So retrieving, say, v1.7.1:Documentation/git-fast-import.txt, would require three round-trips: one to dereference the tag, one to dereference the commit, and then one to finally retrieve the blob.

Another possible concern is that this is very much git specific. Other fast-import backends are just not going to be able to do it with the same format. Is there a convention for naming options like that?

Still, I hope it is useful to start with. Thoughts? Ideas? Improvements?

Jonathan Nieder (3):
  t9300 (fast-import): style tweaks
  Teach fast-import to print the id of each imported commit
  fast-import: Let importers retrieve the objects being written

 Documentation/git-fast-import.txt |   45 +
 fast-import.c                     |  126 ++
 t/t9300-fast-import.sh            | 3371 ++++++++++++++++++++-----------------
 3 files changed, 1960 insertions(+), 1582 deletions(-)