threads / discuss / 17819

Re: git-svn -- out of memory

Subject: Re: git-svn -- out of memory

## tl;dr

3 messages between Feb 16, 2009 and Feb 16, 2009.

replies: 2people: 3as markdown or json

Alfred M. Szmidt· Feb 16, 2009, 09:37 UTC · lore

   > when doing a `git svn clone' on a subversion repository that contains
   > a large file (>1GiB), `git svn' dies with the following:
   > 
   > ams@trillian:~$ git svn clone REPO -T trunk -b branches -t tags
   > [...]
   > fatal: Out of memory? mmap failed: Cannot allocate memory
   > hash-object -w --stdin-paths: command returned error: 128
   > 
   > error closing pipe: Bad file descriptor at /usr/local/bin/libexec/git-core//git-svn line 0
   > error closing pipe: Bad file descriptor at /usr/local/bin/libexec/git-core//git-svn line 0
   > 
   > Doing a `git repack' doesn't help either.  Anyone got a clue how to
   > either solve, or work around this?
   > 
   > git version 1.6.1.3

   Hi Alfred,

   Core git requires objects to be able to fit into (virtual) memory.
   Adding more swap space should solve the problem for you, but this is not
   only a git-svn problem.

In this case, the machine has 32G RAM, including ~20G swap. So I do not think that this is the case, it could be I suppose...

Cheers.

Junio C Hamano· Feb 16, 2009, 10:05 UTC · re: Alfred M. Szmidt · lore

"Alfred M\. Szmidt" <ams@gnu.org> writes:

Show 7 quoted lines

>    > when doing a `git svn clone' on a subversion repository that contains
>    > a large file (>1GiB), `git svn' dies with the following:
>    > 
>    > ams@trillian:~$ git svn clone REPO -T trunk -b branches -t tags
>    > [...]
>    > fatal: Out of memory? mmap failed: Cannot allocate memory
>    > hash-object -w --stdin-paths: command returned error: 128

Ding...

Show 8 quoted lines

>    > git version 1.6.1.3
>
>    Core git requires objects to be able to fit into (virtual) memory.
>    Adding more swap space should solve the problem for you, but this is not
>    only a git-svn problem.
>
> In this case, the machine has 32G RAM, including ~20G swap.  So I do
> not think that this is the case, it could be I suppose...

The hash-object error message sounds like the issue that was fixed with 915308b (avoid 31-bit truncation in write_loose_object, 2009-01-29). Does v1.6.2-rc1 help?

Jeff King· Feb 16, 2009, 13:45 UTC · re: Junio C Hamano · lore

On Mon, Feb 16, 2009 at 02:05:54AM -0800, Junio C Hamano wrote:

Show 15 quoted lines

> "Alfred M\. Szmidt" <ams@gnu.org> writes:
> 
> >    > when doing a `git svn clone' on a subversion repository that contains
> >    > a large file (>1GiB), `git svn' dies with the following:
> >    > 
> >    > ams@trillian:~$ git svn clone REPO -T trunk -b branches -t tags
> >    > [...]
> >    > fatal: Out of memory? mmap failed: Cannot allocate memory
> >    > hash-object -w --stdin-paths: command returned error: 128
> 
> Ding...
> [...]
> The hash-object error message sounds like the issue that was fixed with
> 915308b (avoid 31-bit truncation in write_loose_object, 2009-01-29).  Does
> v1.6.2-rc1 help?

Hmm. The original problem I had with the 32-bit wrapping actually caused malloc to fail, whereas this is an mmap failure. So it might not be the same thing (but certainly testing with v1.6.2-rc1 will be a good data point).

-Peff