Re: git-svn clone -- runs out of memory

From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Date: May 19, 2010, 16:33 UTC
Message-ID: <AANLkTimm_9Ks3adWqXpVBbY6osLl_3c029x8eMI5tb1M@mail.gmail.com>
In-Reply-To: <F47FB1C2-3EEB-475C-8874-CBEC45582386@verizon.net>

On Wed, May 19, 2010 at 05:51, Jack Moore <jhmoore719@verizon.net> wrote:

> When I "git-svn clone" the repository it takes about 30 hour (yes
> hours)  to (appears to) retrieve the files.

That's relatively short actually with git-svn. I've taken 3 weeks to clone one large (~70k revisions + lots of branches) repository, and that was via local file:// access.

> Then git goes into either a packing or garbage collecting phase and
> eventually runs out of memory.  I get an error relating to "mmap
> ... out of memory".  When it runs out of memory the "cloned"
> repository is ~2.4 GBi.

This could be one of a few things:

  * git-svn leaks *lots* of memory.

    The solution is to just kill it and restart it every few
    hours. After it's killed run `git svn fetch`. You should be doing
    this anyway on any clone that takes >1 hour, aside from leaking
    memory it gets a lot slower.

  * The known problem with some git utilities doing things in-memory.

    If that's the case the solution is to just add lots of swap (or
    patch the core).

  * Something I'm forgetting

In any case, run it with GIT_TRACE=1 so you can see what command is the one that failed.