# git-repack keeps running out of memory 5 messages from 2015-06-01 to 2015-06-02. Participants: Phil Susi, Junio C Hamano, Jeff King, Phillip Susi. Thread: https://gitlist.dev/t/39487 ## Phil Susi, 2015-06-01 18:39 Subject: git-repack keeps running out of memory Message-ID: <556CA6C8.6000804@ubuntu.com> URL: https://gitlist.dev/e/556CA6C8.6000804%40ubuntu.com ``` I keep having git-repack run out of virtual memory ( 32 bit system ) when trying to repack my linux kernel repo. It keeps making it right up to 99% then barfing saying mmap failed: Cannot allocate memory. I thought I could help this by limiting the pack size, and using --window-memory to limit the memory usage, but it still happens with this full command line: git repack -a -d --max-pack-size=500m -f -F --depth=20 --window=250 --window-memory=500m The key factor seems to be the --window... with 50 it works fine, but with 100 or more, even with the very low --window-memory limit, it crashes. ``` ## Junio C Hamano, 2015-06-01 18:43 Subject: Re: git-repack keeps running out of memory Message-ID: URL: https://gitlist.dev/e/xmqqlhg35ky2.fsf%40gitster.dls.corp.google.com In-Reply-To: <556CA6C8.6000804@ubuntu.com> ``` Phil Susi writes: > I keep having git-repack run out of virtual memory ( 32 bit system ) > when trying to repack my linux kernel repo. It keeps making it right > up to 99% then barfing saying mmap failed: Cannot allocate memory. > > I thought I could help this by limiting the pack size, and using > --window-memory to limit the memory usage, but it still happens with > this full command line: > > git repack -a -d --max-pack-size=500m -f -F --depth=20 --window=250 > --window-memory=500m > > The key factor seems to be the --window... with 50 it works fine, but > with 100 or more, even with the very low --window-memory limit, it > crashes. Unfortunately, that is totally expected. Window tells us to keep enough information to compare with that many objects in-core. I do not think max-pack-size would affect much. ``` ## Phil Susi, 2015-06-01 19:07 Subject: Re: git-repack keeps running out of memory Message-ID: <556CAD64.8000208@ubuntu.com> URL: https://gitlist.dev/e/556CAD64.8000208%40ubuntu.com In-Reply-To: ``` On 6/1/2015 2:43 PM, Junio C Hamano wrote: > Phil Susi writes: > >> I keep having git-repack run out of virtual memory ( 32 bit system ) >> when trying to repack my linux kernel repo. It keeps making it right >> up to 99% then barfing saying mmap failed: Cannot allocate memory. >> >> I thought I could help this by limiting the pack size, and using >> --window-memory to limit the memory usage, but it still happens with >> this full command line: >> >> git repack -a -d --max-pack-size=500m -f -F --depth=20 --window=250 >> --window-memory=500m >> >> The key factor seems to be the --window... with 50 it works fine, but >> with 100 or more, even with the very low --window-memory limit, it >> crashes. > > Unfortunately, that is totally expected. Window tells us to keep > enough information to compare with that many objects in-core. I do > not think max-pack-size would affect much. It's more the --window-memory argument that is important here: it is supposed to prevent exactly this problem. I guess I tried adding the --max-pack-size as well on the off chance that it would help. ``` ## Jeff King, 2015-06-01 20:58 Subject: Re: git-repack keeps running out of memory Message-ID: <20150601205854.GA21144@peff.net> URL: https://gitlist.dev/e/20150601205854.GA21144%40peff.net In-Reply-To: <556CAD64.8000208@ubuntu.com> ``` On Mon, Jun 01, 2015 at 03:07:16PM -0400, Phil Susi wrote: > It's more the --window-memory argument that is important here: it is > supposed to prevent exactly this problem. I guess I tried adding the > --max-pack-size as well on the off chance that it would help. How many processors do you have? The window-memory is per-thread. Try with --threads=1. -Peff ``` ## Phillip Susi, 2015-06-02 00:04 Subject: Re: git-repack keeps running out of memory Message-ID: <556CF306.6000308@ubuntu.com> URL: https://gitlist.dev/e/556CF306.6000308%40ubuntu.com In-Reply-To: <20150601205854.GA21144@peff.net> ``` -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On 06/01/2015 04:58 PM, Jeff King wrote: > How many processors do you have? The window-memory is per-thread. > Try with --threads=1. Ahh, right... 8... that explains it ;) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBCgAGBQJVbPMBAAoJENRVrw2cjl5R2rkH/0qa9HerYM7Bw5qfcV3zahFz 5g/NGoW3M9IutqqLiyDzAQboj62+TzrEuM/fo7BgZrKYBsA7/4SNdJMmabaUb5Kr rtW1qfpErIHUQUY0wAxBwXX7gpihB3HTYRsZAX+7Itfae7bw8r3krj/L1Ty2nj3Y QLIm9uwijiBNpHoF9Jnl6gbklaVFbBBfdGH2uPLzoso5YdlJ0L6o4VDmeD55njjx 30cOLDwInQ4zxdGKBH1SaIXGq+2Vi1txLrzYaaEOA+NW9+Rp7J9Anqcv/IH5yyX/ y9H0yDXXP/3gPwTHvYiRTVWMjAvjxSdc4n4ZgySLBPTruW1utxjPdaTSffm2xsg= =wza7 -----END PGP SIGNATURE----- ```