Re: Incremental cvsimports

From: Geoff Russell <geoffrey.russell@gmail.com>
Date: May 24, 2006, 11:19 UTC
Message-ID: <93c3eada0605240419o48891cdle6c100fc0ac870ff@mail.gmail.com>
In-Reply-To: <46a038f90605240121o117fadb6vf3ce910a3ad3e90@mail.gmail.com>

Dear Martin,

On 5/24/06, Martin Langhoff <martin.langhoff@gmail.com> wrote:

Show 15 quoted lines

> On 5/24/06, Geoff Russell <geoffrey.russell@gmail.com> wrote:
> > Dear Git,
>
> Dear Geoff,
>
> if you look at the list archive for the last couple of days, you'll
> see there's been quite a bit of activity in tuning cvsimport so that
> it scales better with large imports like yours. We have been playing
> with a gentoo cvs repo with 300K commits / 1.6GB uncompressed.
>
> Don't split up the tree... that'll lead to something rather ackward.
> Instead, fetch and build git from Junio's 'master' branch which seems
> to have collected most (all?) of the patches posted, including one
> from Linus that will repack the repo every 1K commits -- keeping the
> import size down.

I got the latest git and yes, the size is kept down. I've only tried with a smaller repository but it looks promising. When I ran git-cvsimport without a CVS-module name (wanting the entire repository), it gave me a Usage message indicating that the CVS-module name was optional - but it isn't :)

I did have to change 2 lines in git-cvsimport to get it to run with my 5.8.0 perl (problems with POSIX errno). I've attached a patch but my work around isn't as quick as what it replaced.

Many thanks, I'll have a go with the big repository at work tomorrow!

Cheers, Geoff Russell

P.S. I've just started to look with git. We have wanted a cvs replacement for a while but have been too scared to change (until now).

Show 6 quoted lines

>
> You _will_ need a lot of memory though, as cvsps grows large (working
> on a workaround now) and cvsimport grows a bit over time (where is
> that last leak?!). And a fast machine -- specially fast IO. I've just
> switched from an old test machine to an AMD64 with fast disks, and
> it's importing around 10K commits per hour.

Show 9 quoted lines

>
> You will probably want to run cvsps by hand, and later use the -P flag.
>
> cheers,
>
>
> martin
>
>