{"thread":{"id":"23841","subject":"git-svn clone -- runs out of memory","startedAt":"2010-05-19T05:51:27Z","lastAt":"2010-05-19T16:33:41Z","messageCount":2,"participants":["Jack Moore","Ævar Arnfjörð Bjarmason"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"141898","messageId":"F47FB1C2-3EEB-475C-8874-CBEC45582386@verizon.net","threadId":"23841","inReplyTo":null,"subject":"git-svn clone -- runs out of memory","fromName":"Jack Moore","fromEmail":"jhmoore719@verizon.net","sentAt":"2010-05-19T05:51:27Z","receivedAt":"2010-05-19T05:51:27Z","isPatch":false,"sender":{"key":"jhmoore719@verizon.net","avatar":null},"body":"I'm trying to transition from subversion to git.  I have a very poorly structured subversion repository.  It has multiple (dozens) copies of many (thousands) of binary files that don't change from version to version.  When I \"git-svn clone\" the repository it takes about 30 hour (yes hours)  to (appears to) retrieve the files.  Then git goes into either a packing or garbage collecting phase and eventually runs out of memory.  I get an error relating to \"mmap ... out of memory\".  When it runs out of memory the \"cloned\" repository is ~2.4 GBi.\n\nI don't know what files it's failing on.  I've tried excluding the directories that contain the binary files.  Does anyone have suggestions on isolating the problem or working around the memory limitation?\n\nThanks\nJack"},{"id":"141912","messageId":"AANLkTimm_9Ks3adWqXpVBbY6osLl_3c029x8eMI5tb1M@mail.gmail.com","threadId":"23841","inReplyTo":"F47FB1C2-3EEB-475C-8874-CBEC45582386@verizon.net","subject":"Re: git-svn clone -- runs out of memory","fromName":"Ævar Arnfjörð Bjarmason","fromEmail":"avarab@gmail.com","sentAt":"2010-05-19T16:33:41Z","receivedAt":"2010-05-19T16:33:41Z","isPatch":false,"sender":{"key":"avarab@gmail.com","avatar":"https://avatars.githubusercontent.com/u/45301?v=4"},"body":"On Wed, May 19, 2010 at 05:51, Jack Moore <jhmoore719@verizon.net> wrote:\n\n> When I \"git-svn clone\" the repository it takes about 30 hour (yes\n> hours)  to (appears to) retrieve the files.\n\nThat's relatively short actually with git-svn. I've taken 3 weeks to\nclone one large (~70k revisions + lots of branches) repository, and\nthat was via local file:// access.\n\n> Then git goes into either a packing or garbage collecting phase and\n> eventually runs out of memory.  I get an error relating to \"mmap\n> ... out of memory\".  When it runs out of memory the \"cloned\"\n> repository is ~2.4 GBi.\n\nThis could be one of a few things:\n\n  * git-svn leaks *lots* of memory.\n\n    The solution is to just kill it and restart it every few\n    hours. After it's killed run `git svn fetch`. You should be doing\n    this anyway on any clone that takes >1 hour, aside from leaking\n    memory it gets a lot slower.\n\n  * The known problem with some git utilities doing things in-memory.\n\n    If that's the case the solution is to just add lots of swap (or\n    patch the core).\n\n  * Something I'm forgetting\n\nIn any case, run it with GIT_TRACE=1 so you can see what command is\nthe one that failed.\n"}]}