From: Shawn Pearce Date: Sun, 15 May 2011 20:37:27 GMT Subject: Re: how to keep git-fetch from running out of memory? Message-ID: In-Reply-To: <7vboz3pv7x.fsf@alter.siamese.dyndns.org> On Sun, May 15, 2011 at 13:25, Junio C Hamano wrote: > Shawn Pearce writes: > >> However, that may still be insufficient. A big object must still be >> allocated in memory in order to compute its SHA-1. > > I wonder if we can stream this? If its a non-delta, yes. We already do this in JGit for blobs over 50M. (Trees, commits and tags are not streamed because we might need to do fsck on them and the fsck code for these object types requires the entire object in a single buffer.) Its probably not a huge change to index-pack.c. But I haven't dug around in there in a while. I wish I could say I will look at this on Monday, but I don't have the time. -- Shawn.