{"thread":{"id":"17822","subject":"Re: git-svn -- out of memory","startedAt":"2009-02-16T14:23:40Z","lastAt":"2009-02-16T15:37:45Z","messageCount":2,"participants":["Alfred M. Szmidt","Jeff King"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"104948","messageId":"1234794220.676511.4320.nullmailer@beryx.hq.kred","threadId":"17822","inReplyTo":null,"subject":"Re: git-svn -- out of memory","fromName":"Alfred M. Szmidt","fromEmail":"ams@gnu.org","sentAt":"2009-02-16T14:23:40Z","receivedAt":"2009-02-16T14:23:40Z","isPatch":false,"sender":{"key":"ams@gnu.org","avatar":"https://gravatar.com/avatar/46825633bc1d168c418bbf90c7859b9f83888b45d7cc546a8544df495858df83?d=mp&s=160"},"body":"   >    > when doing a `git svn clone' on a subversion repository that contains\n   >    > a large file (>1GiB), `git svn' dies with the following:\n   >    > \n   >    > ams@trillian:~$ git svn clone REPO -T trunk -b branches -t tags\n   >    > [...]\n   >    > fatal: Out of memory? mmap failed: Cannot allocate memory\n   >    > hash-object -w --stdin-paths: command returned error: 128\n\n   Ding...\n\n   >    > git version 1.6.1.3\n   >\n   >    Core git requires objects to be able to fit into (virtual) memory.\n   >    Adding more swap space should solve the problem for you, but this is not\n   >    only a git-svn problem.\n   >\n   > In this case, the machine has 32G RAM, including ~20G swap.  So I do\n   > not think that this is the case, it could be I suppose...\n\n   The hash-object error message sounds like the issue that was fixed with\n   915308b (avoid 31-bit truncation in write_loose_object, 2009-01-29).  Does\n   v1.6.2-rc1 help?\n\nams@trillian:~/# git --version\ngit version 1.6.2.rc1\nams@trillian:~/# git svn clone REPO -T trunk -b branches -t tags\n[...]\nfatal: Out of memory? mmap failed: Cannot allocate memory\nhash-object -w --stdin-paths: command returned error: 128\n\nNope, behaves the same way.\n\nI'll try adding more swap and see what happens...\n\nThanks for the help.\n"},{"id":"104957","messageId":"20090216153745.GA22886@coredump.intra.peff.net","threadId":"17822","inReplyTo":"1234794220.676511.4320.nullmailer@beryx.hq.kred","subject":"Re: git-svn -- out of memory","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2009-02-16T15:37:45Z","receivedAt":"2009-02-16T15:37:45Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Feb 16, 2009 at 03:23:40PM +0100, Alfred M. Szmidt wrote:\n\n> ams@trillian:~/# git --version\n> git version 1.6.2.rc1\n> ams@trillian:~/# git svn clone REPO -T trunk -b branches -t tags\n> [...]\n> fatal: Out of memory? mmap failed: Cannot allocate memory\n> hash-object -w --stdin-paths: command returned error: 128\n> \n> Nope, behaves the same way.\n\nHmm. It would help to know what object it is failing on. I assume you\nrepo is too large to be easily shared? If not, can you try instrumenting\nhash-object with the patch below and seeing which file it is barfing on?\n\nI'd be curious to see the size of the file (in case it is >2G and we\nhave another signed integer problem) and its position in the list (in\ncase we are leaking memory and you are hitting a system limit).\n\n---\ndiff --git a/hash-object.c b/hash-object.c\nindex 37e6677..20371d0 100644\n--- a/hash-object.c\n+++ b/hash-object.c\n@@ -44,6 +44,7 @@ static void hash_stdin_paths(const char *type, int write_objects)\n \t\t\t\tdie(\"line is badly quoted\");\n \t\t\tstrbuf_swap(&buf, &nbuf);\n \t\t}\n+\t\tfprintf(stderr, \"debug: hashing %s\\n\", buf.buf);\n \t\thash_object(buf.buf, type, write_objects, buf.buf);\n \t}\n \tstrbuf_release(&buf);\n"}]}