From: Linus Torvalds Date: Fri, 05 Jan 2007 23:17:30 GMT Subject: Re: git-svnimport failed and now git-repack hates me Message-ID: In-Reply-To: <7v64bldqas.fsf@assigned-by-dhcp.cox.net> On Fri, 5 Jan 2007, Junio C Hamano wrote: > > The problem Linus pointed out was that your SHA1_Update() > implementations may not be prepared to hash the whole 2.3GB in > one go. The one in "master" (and "maint", although I haven't > done a v1.4.4.4 maintenance release yet) calls SHA1_Update() > in chunks to work around that potential issue. Well, I think Chris is worried about having it all mapped at the same time. It does actually end up forcing the kernel to do more work (it's harder to re-use a mapped page than it is to reuse one that isn't), and in that sense, if you have less than GB of RAM and can't just keep it all in memory at the same time, doing one large mmap is possibly more expensive than chunking things up. That said, I doubt it's a huge problem. If you can't fit the whole file in memory, your real performance issue is going to be the IO, not the fact that the kernel has to work a bit harder at unmapping pages ;) Linus