From: Christopher Jefferson Date: Wed, 27 May 2009 13:02:29 GMT Subject: Re: Problem with large files on different OSes Message-ID: In-Reply-To: <4A1D2614.5060303@op5.se> On 27 May 2009, at 12:37, Andreas Ericsson wrote: > Christopher Jefferson wrote: >> I recently came across a very annoying problem, characterised by >> the following example: >> On a recent ubuntu install: >> dd if=/dev/zero of=file bs=1300k count=1k >> git commit file -m "Add huge file" >> The repository can be pulled and pushed successfully to other >> ubuntu installs, but on Mac OS X, 10.5.7 machine with 4GB ram git >> pull produces: >> remote: Counting objects: 6, done. >> remote: git(1533,0xb0081000) malloc: *** mmap(size=1363152896) >> failed (error code=12) >> remote: *** error: can't allocate region >> remote: *** set a breakpoint in malloc_error_break to debug >> remote: git(1533,0xb0081000) malloc: *** mmap(size=1363152896) >> failed (error code=12) >> remote: *** error: can't allocate region >> remote: *** set a breakpoint in malloc_error_break to debug >> remote: fatal: Out of memory, malloc failed >> error: git upload-pack: git-pack-objects died with error. >> fatal: git upload-pack: aborting due to possible repository >> corruption on the remote side. >> remote: aborting due to possible repository corruption on the >> remote side. >> fatal: protocol error: bad pack header >> The problem appears to be the different maximum mmap sizes >> available on different OSes. Whic I don't really mind the maximum >> file size restriction git imposes, this restriction varying from OS >> to OS is very annoying, fixing this required rewriting history to >> remove the commit, which caused problems as the commit had already >> been pulled, and built on, by a number of developers. >> If the requirement that all files can be mmapped cannot be easily >> removed, would be it perhaps be acceptable to impose a (soft?) >> 1GB(ish) file size limit? > > Most definitely not. Why should we limit a cross-platform system for > the benefit of one particular developer's lacking hardware? Out of curiosity, why do you say lacking hardware? I am running ubuntu, windows and Mac OS X on exactly the same machine, which is not running out of physical memory, never mind swap, when using git on any OS. The problem is purely a software (and OS) problem. Chris