Re: [WIP PATCH] Add 'git fast-export', the sister of 'git fast-import'

From: Han-Wen Nienhuys <hanwenn@gmail.com>
Date: Nov 23, 2007, 01:23 UTC
Message-ID: <f329bf540711221723g2754ce03r4da6d429c45668c@mail.gmail.com>
In-Reply-To: <Pine.LNX.4.64.0711230050270.27959@racer.site>

2007/11/22, Johannes Schindelin <Johannes.Schindelin@gmx.de>:

Show 9 quoted lines

> > The cool thing about git-fast-import is that it reads from stdin, has a
> > very easy to use programmatic interface, and does not impose any order
> > on how you enter the information.
> >
> > This doesn't seem to be mirrored by this script?
>
> Umm.  What exactly do you want to reorder?  I mean, this program is meant
> to dump the complete contents to stdout (whether you redirect it to a file
> or edit it with sed does not concern this program).  It does that.

fast-import does not need to take a complete repository as input, but can also update lots of branches incrementally, all from one process. Likewise, it would be nice to have a program that can also dump small bits of information.

Maybe I'm misunderstanding you, but fast-export just does not seem a mirror of fast-import; perhaps you can name it 'dump-all' or something?

> Maybe you want to specify if all blobs should be output first, and then
> the commits?  Or files should be used?  But all of these things seem to be
> useless to me.

No, I want the program to wait for me to tell it what blobs/commits/trees I want. The commit I want to see secondly may depend on the output I read in the first request blob. Right now, for each data dependency I have to start a new git process.

Show 6 quoted lines

> > Besides being a nuisance, I actually run git on NFS, and every git
> > process has to go to NFS a couple times to retrieve the same
> > information. This has a noticeable performance impact.
>
> Why don't you just work on a local clone?  If it is really performance
> critical, and I/O is an issue, you are better off working in a tmpfs.

In a company setting, NFS is the easiest way to share information with colleagues without breaking access control and making our security staff nervous. It's also snapshotted and backed up automatically.

Show 11 quoted lines

> > It would make my life a lot easier if I could simply open a pipe to a
> > single process for the duration of the script, and do all my queries to
> > this one process.  Of course, if the repository is changed by another
> > process, I would have to restart it, but that's manageable.  I could
> > even write a nice Python class that runs both fast-import and
> > fast-export. I could then have an efficient Python interface to a
> > git-repository, without needing any library wrapping.
>
> There is a minimal python wrapper to libgit-thin, which was one of our
> GSoC projects.  You might want to look at this if you are really that
> unhappy with the existing framework.

What's the status of this?  I prefer not to diverge from mainline too much.

-- 
Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen