{"thread":{"id":"28390","subject":"git gc exit with out of memory, malloc failed error","startedAt":"2011-09-15T01:33:42Z","lastAt":"2011-09-15T09:16:13Z","messageCount":5,"participants":["Alexander Kostikov","David Michael Barr","Brandon Casey","Carlos Martín Nieto"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"175531","messageId":"CAGAhT3mbGB-0q3EKh5MrGqB59wUea7NfaaY18DvnL3qimwh9QA@mail.gmail.com","threadId":"28390","inReplyTo":null,"subject":"git gc exit with out of memory, malloc failed error","fromName":"Alexander Kostikov","fromEmail":"alex.kostikov@gmail.com","sentAt":"2011-09-15T01:33:42Z","receivedAt":"2011-09-15T01:33:42Z","isPatch":false,"sender":{"key":"alex.kostikov@gmail.com","avatar":"https://gravatar.com/avatar/790c4ea9bf7f65be03b0d1c389b5dee8d1e704441d75e690c4372c30f7943e14?d=mp&s=160"},"body":"I'm new to git and I'm getting the following out of memory error on git gc:\n\n$ git gc\nCounting objects: 80818, done.\nDelta compression using up to 8 threads.\nfatal: Out of memory, malloc failed (tried to allocate 24359675 bytes)\nerror: failed to run repack\n\nThe only advice I found in the internet suggested to run repack with\n--window-memory parameter specified. But this call also fails:\n\n$ git repack -adf --window-memory=0\nCounting objects: 80818, done.\nDelta compression using up to 8 threads.\nwarning: suboptimal pack - out of memory\nfatal: Out of memory, malloc failed (tried to allocate 24356363 bytes)\n\nHow do I cleanup my repository?\n\n$ git version\ngit version 1.7.6.msysgit.0\n\nOS: Windows Server 2008 R2 SP1 (x64)\nPhysical memory: 24 GB\nThe commands listed were executed under x64 console process.\n\n-- \nThanks,\nAlexander Kostikov\n"},{"id":"175538","messageId":"CAFfmPPP58J90758TRB2sAj-Wr5HB=rQtaJipKovsHXyTcswuJQ@mail.gmail.com","threadId":"28390","inReplyTo":"CAGAhT3mbGB-0q3EKh5MrGqB59wUea7NfaaY18DvnL3qimwh9QA@mail.gmail.com","subject":"Re: git gc exit with out of memory, malloc failed error","fromName":"David Michael Barr","fromEmail":"davidbarr@google.com","sentAt":"2011-09-15T02:20:16Z","receivedAt":"2011-09-15T02:20:16Z","isPatch":false,"sender":{"key":"davidbarr@google.com","avatar":"https://avatars.githubusercontent.com/u/220594?v=4"},"body":"On Thu, Sep 15, 2011 at 11:33 AM, Alexander Kostikov\n wrote:\n> I'm new to git and I'm getting the following out of memory error on git gc:\n>\n> $ git gc\n> Counting objects: 80818, done.\n> Delta compression using up to 8 threads.\n> fatal: Out of memory, malloc failed (tried to allocate 24359675 bytes)\n> error: failed to run repack\n>\n> The only advice I found in the internet suggested to run repack with\n> --window-memory parameter specified. But this call also fails:\n>\n> $ git repack -adf --window-memory=0\n> Counting objects: 80818, done.\n> Delta compression using up to 8 threads.\n> warning: suboptimal pack - out of memory\n> fatal: Out of memory, malloc failed (tried to allocate 24356363 bytes)\n>\n> How do I cleanup my repository?\n>\n> $ git version\n> git version 1.7.6.msysgit.0\n>\n> OS: Windows Server 2008 R2 SP1 (x64)\n> Physical memory: 24 GB\n> The commands listed were executed under x64 console process.\n>\n> --\n> Thanks,\n> Alexander Kostikov\n> --\n> To unsubscribe from this list: send the line \"unsubscribe git\" in\n> the body of a message to majordomo@vger.kernel.org\n> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>\n\nHi,\n\nMy understanding is that msysgit is a 32-bit build.\nSo if your existing pack is ~2GB, repack will fail.\nAlso, I think that setting window-memory to 0\nmeans no limit, which is not what you want.\nOne value I have seen suggested is 256m.\nIn my experience, peak memory consumption of\nrepack is proportional to the value of --window,\nso you might want to try tweaking that.\nThere are quite a few config parameters that\naffect the memory consumption of repack.\n\n-- \nDavid Barr\n"},{"id":"175540","messageId":"CA+sFfMdXLkSE_RXB28TSx0JpPVjwe1Rf+03TkiT2YJjYH7m9MQ@mail.gmail.com","threadId":"28390","inReplyTo":"CA+sFfMcfy=GCFrCjonQXvXRQu=hLjDvQViJJ75xqa72Gb23MgQ@mail.gmail.com","subject":"Re: git gc exit with out of memory, malloc failed error","fromName":"Brandon Casey","fromEmail":"drafnel@gmail.com","sentAt":"2011-09-15T02:29:09Z","receivedAt":"2011-09-15T02:29:09Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"[resend since gmail's Rich formatting was enabled]\n\nOn Wed, Sep 14, 2011 at 8:33 PM, Alexander Kostikov\n wrote:\n>\n> I'm new to git and I'm getting the following out of memory error on git gc:\n>\n> $ git gc\n> Counting objects: 80818, done.\n> Delta compression using up to 8 threads.\n> fatal: Out of memory, malloc failed (tried to allocate 24359675 bytes)\n> error: failed to run repack\n\nTry reducing the number of threads that are used.  You must have some\npretty large objects if you have 24GB and ran out of memory.  The\nfollowing will configure git to use only one thread.\n   git config pack.threads=1\n-Brandon\n"},{"id":"175541","messageId":"CAGAhT3ksjRmGVpd2p+q9EjC58xgermTDHsOS64_N_eKDWXyXJQ@mail.gmail.com","threadId":"28390","inReplyTo":"CAFfmPPP58J90758TRB2sAj-Wr5HB=rQtaJipKovsHXyTcswuJQ@mail.gmail.com","subject":"Re: git gc exit with out of memory, malloc failed error","fromName":"Alexander Kostikov","fromEmail":"alex.kostikov@gmail.com","sentAt":"2011-09-15T02:42:18Z","receivedAt":"2011-09-15T02:42:18Z","isPatch":false,"sender":{"key":"alex.kostikov@gmail.com","avatar":"https://gravatar.com/avatar/790c4ea9bf7f65be03b0d1c389b5dee8d1e704441d75e690c4372c30f7943e14?d=mp&s=160"},"body":"Thanks!\n--window-memory=50m worked for my project. 256m was still throwing\nerror. Probably real memory consumption is close to threads number *\nwindow memory.\n\n\nOn Wed, Sep 14, 2011 at 7:20 PM, David Michael Barr\n wrote:\n> On Thu, Sep 15, 2011 at 11:33 AM, Alexander Kostikov\n> wrote:\n>> I'm new to git and I'm getting the following out of memory error on git gc:\n>>\n>> $ git gc\n>> Counting objects: 80818, done.\n>> Delta compression using up to 8 threads.\n>> fatal: Out of memory, malloc failed (tried to allocate 24359675 bytes)\n>> error: failed to run repack\n>>\n>> The only advice I found in the internet suggested to run repack with\n>> --window-memory parameter specified. But this call also fails:\n>>\n>> $ git repack -adf --window-memory=0\n>> Counting objects: 80818, done.\n>> Delta compression using up to 8 threads.\n>> warning: suboptimal pack - out of memory\n>> fatal: Out of memory, malloc failed (tried to allocate 24356363 bytes)\n>>\n>> How do I cleanup my repository?\n>>\n>> $ git version\n>> git version 1.7.6.msysgit.0\n>>\n>> OS: Windows Server 2008 R2 SP1 (x64)\n>> Physical memory: 24 GB\n>> The commands listed were executed under x64 console process.\n>>\n>> --\n>> Thanks,\n>> Alexander Kostikov\n>> --\n>> To unsubscribe from this list: send the line \"unsubscribe git\" in\n>> the body of a message to majordomo@vger.kernel.org\n>> More majordomo info at  http://vger.kernel.org/majordomo-info.html\n>>\n>\n> Hi,\n>\n> My understanding is that msysgit is a 32-bit build.\n> So if your existing pack is ~2GB, repack will fail.\n> Also, I think that setting window-memory to 0\n> means no limit, which is not what you want.\n> One value I have seen suggested is 256m.\n> In my experience, peak memory consumption of\n> repack is proportional to the value of --window,\n> so you might want to try tweaking that.\n> There are quite a few config parameters that\n> affect the memory consumption of repack.\n>\n> --\n> David Barr\n>\n\n\n\n--\nYour sincerely,\nAlexander Kostikov\n"},{"id":"175562","messageId":"1316078174.2019.15.camel@bee.lab.cmartin.tk","threadId":"28390","inReplyTo":"CA+sFfMdXLkSE_RXB28TSx0JpPVjwe1Rf+03TkiT2YJjYH7m9MQ@mail.gmail.com","subject":"Re: git gc exit with out of memory, malloc failed error","fromName":"Carlos Martín Nieto","fromEmail":"carlos@cmartin.tk","sentAt":"2011-09-15T09:16:13Z","receivedAt":"2011-09-15T09:16:13Z","isPatch":false,"sender":{"key":"carlos@cmartin.tk","avatar":"https://gravatar.com/avatar/956bfe8371004f2960febf266a6af789f60cdc01fbae48bb151ad4c9b532c3a2?d=mp&s=160"},"body":"On Wed, 2011-09-14 at 21:29 -0500, Brandon Casey wrote:\n> [resend since gmail's Rich formatting was enabled]\n> \n> On Wed, Sep 14, 2011 at 8:33 PM, Alexander Kostikov\n> wrote:\n> >\n> > I'm new to git and I'm getting the following out of memory error on git gc:\n> >\n> > $ git gc\n> > Counting objects: 80818, done.\n> > Delta compression using up to 8 threads.\n> > fatal: Out of memory, malloc failed (tried to allocate 24359675 bytes)\n> > error: failed to run repack\n> \n> Try reducing the number of threads that are used. You must have some\n> pretty large objects if you have 24GB and ran out of memory. The\n\nIIRC the 32-bit Windows builds limit each user process to 2GB. If\nmsysgit is built in 32-bit mode, it's possible (probable even, in order\nto remain backwards-compatible) that it's hitting this limit, rather\nthan the physical limit of the machine. If it's using 8 threads, it\nmeans that there are 256MB available for each thread.\n\n cmn\n"}]}