Re: Problem with large files on different OSes
- From
- Christopher Jefferson <caj@cs.st-andrews.ac.uk>
- Date
- May 27, 2009, 13:02 UTC
- Message-ID
- <submission.1M9Im6-0003Hs-40@mail.cs.st-andrews.ac.uk>
- In-Reply-To
- <4A1D2614.5060303@op5.se>
On 27 May 2009, at 12:37, Andreas Ericsson wrote:
Show 37 quoted lines
> Christopher Jefferson wrote: >> I recently came across a very annoying problem, characterised by >> the following example: >> On a recent ubuntu install: >> dd if=/dev/zero of=file bs=1300k count=1k >> git commit file -m "Add huge file" >> The repository can be pulled and pushed successfully to other >> ubuntu installs, but on Mac OS X, 10.5.7 machine with 4GB ram git >> pull produces: >> remote: Counting objects: 6, done. >> remote: git(1533,0xb0081000) malloc: *** mmap(size=1363152896) >> failed (error code=12) >> remote: *** error: can't allocate region >> remote: *** set a breakpoint in malloc_error_break to debug >> remote: git(1533,0xb0081000) malloc: *** mmap(size=1363152896) >> failed (error code=12) >> remote: *** error: can't allocate region >> remote: *** set a breakpoint in malloc_error_break to debug >> remote: fatal: Out of memory, malloc failed >> error: git upload-pack: git-pack-objects died with error. >> fatal: git upload-pack: aborting due to possible repository >> corruption on the remote side. >> remote: aborting due to possible repository corruption on the >> remote side. >> fatal: protocol error: bad pack header >> The problem appears to be the different maximum mmap sizes >> available on different OSes. Whic I don't really mind the maximum >> file size restriction git imposes, this restriction varying from OS >> to OS is very annoying, fixing this required rewriting history to >> remove the commit, which caused problems as the commit had already >> been pulled, and built on, by a number of developers. >> If the requirement that all files can be mmapped cannot be easily >> removed, would be it perhaps be acceptable to impose a (soft?) >> 1GB(ish) file size limit? > > Most definitely not. Why should we limit a cross-platform system for > the benefit of one particular developer's lacking hardware?
Out of curiosity, why do you say lacking hardware? I am running ubuntu, windows and Mac OS X on exactly the same machine, which is not running out of physical memory, never mind swap, when using git on any OS. The problem is purely a software (and OS) problem.
Chris