Re: [PATCH 5/6] Introduce the git fast-import-helper

From: Sverre Rabbelier <srabbelier@gmail.com>
Date: Oct 3, 2010, 23:15 UTC
Message-ID: <AANLkTi=RASZU2e+WV6kXnUH=afE=g2SoGuJFZF1QJ4=D@mail.gmail.com>
In-Reply-To: <4CA8BFB7.2050707@dbservice.com>

Heya,

On Sun, Oct 3, 2010 at 19:39, Tomas Carnecky <tom@dbservice.com> wrote:

Show 6 quoted lines

> I also use a hash table (struct hash_table from hash.h). It's indexed by
> the atom. So it's about equally fast as the existing one but uses
> slightly more memory. I measured the speed and fih is about 5% slower
> than fi. Also, I found out that setting max_packfile to 32MB makes the
> import much faster (from 10 minutes down to 3m to import the sources of
> git itself).

That's curious.

A 5% increase is significant. Imagine importing netbeans, (which is actually a use case of git-remote-hg), which takes about 4 hours. A 5% slowdown means the process will take more than 10 minutes longer.

Show 6 quoted lines

> I do: fprintf(stdout, "mark :%s %s\n", mark, sha1_to_hex(sha1));
> One reason why not just write the plain hash is because that's the same
> syntax as the fih accepts in its input. This way you can do:
>  $ ( cat marks; cat fast-export-stream ) | git fast-import-helper >> marks
> and can restart at any time. Also, making the output a bit more
> structured allows it to be easily extended in the future.

I don't see much benefit tbh, if we want to do something like that it could (relatively) easy be added to regular git fast-import with a feature. So you'd start the stream with "feature new-marks-format" and only then follow up with "feature import-marks=...". Ditto on the commandline, `git fast-import --new-marks-format --import-marks=...". If it turns out to be very useful/popular it can be made the default after warning for a full release first.

-- 
Cheers,

Sverre Rabbelier