From: Thomas Date: Wed, 09 May 2012 09:36:10 GMT Subject: Re: Large repo and pack.packsizelimit Message-ID: In-Reply-To: Nicolas Pitre fluxnic.net> writes: > > On Tue, 8 May 2012, Jeff King wrote: > > > On Tue, May 08, 2012 at 05:13:13PM -0400, Nicolas Pitre wrote: > > > > > > This should be fixed in git. Unfortunately, I don't know that it is as > > > > trivial as just splitting the incoming stream; we would also have to > > > > make sure that there were no cross-pack deltas in the result. > > > > > > IMHO this is the wrong fix. The pack size limit was created to deal > > > with storage media with limited capacity. In this case, the repack > > > process should be told to limit its memory usage, and pack-index should > > > simply be taught to cope. > > > > Hmm, you're right. I was thinking it helped to deal with memory > > addressing issues for 32-bit systems, but I guess > > core.packedGitWindowSize should be handling that. IOW, the 10G packfile > > should work just fine for normal access. > > > > However, the OP did say he got an "out of memory" error during the > > clone. So maybe there is a problem to be fixed in index-pack there. > > Was the OOM on the remote side (pack-objects) or on the local side > (index-pack) ? > > Nicolas > To be exact I did the clone locally on the same machine and so the clone itself worked but I got the OOM during the first fetch. I "fixed" this by setting transfer.unpacklimit=100000 which caused only loose objects to be transfered. So in this case I think the OOM was on the remote side. But there is another OOM if I try to repack locally. It seems to me that neither pack-objects nor index-pack respekt pack.packsizelimit and always try to pack all objects to be transferred resp. all local loose objects in one pack. I could live wth the transfer.unpacklimit=100000 but the local OOM stops me from using the cloned repo. --- Thomas