Re: Git exhausts memory.
- From
- Alif Wahid <alif.wahid@gmail.com>
- Date
- Apr 5, 2011, 04:35 UTC
- Message-ID
- <BANLkTik6XoWaehp7=5fxwcyhkTHWCO8-Eg@mail.gmail.com>
- In-Reply-To
- <20110405022235.GA4414@spacedout.fries.net>
Hi everyone,
On 5 April 2011 12:22, David Fries <david@fries.net> wrote:
Show 25 quoted lines
> On Mon, Apr 04, 2011 at 09:57:01PM +0700, Nguyen Thai Ngoc Duy wrote: >> On Mon, Apr 4, 2011 at 7:52 PM, Alif Wahid <alif.wahid@gmail.com> wrote: >> > Hi Nicolas, >> > >> > On 4 April 2011 01:18, Nicolas Pitre <nico@fluxnic.net> wrote: >> >> >> >> Something you can try is to simply tell Git not to attempt any delta >> >> compression on those tar files using gitattributes (see the man page of >> >> the same name). >> >> Should we change the default to not delta if a blob exceeds predefined >> limit (say 128M)? People who deliberately wants to delta them can >> still set delta attr. 1.8.0 material maybe? > > I think it would be better to define it in terms of available memory. > Something like the minimum of system memory or address space, and > delta up to X amount of that (it might be good to leave off swap to > reduce trashing). There has to be a better way than a straight 128MB > default. > > The number which works on my 8GB desktop is going to kill the computer > in the trunk of my car with 48MB of ram. I've actually seen a 700 MB > repository fail with `git-gc --aggressive` on a system with 4GB of ram > because it ran out of memory, it only worked by leaving off the > --aggressive option.
It seems to me that if "git init" creates a $GIT_DIR/info/attributes file by default with a line like "*.gz -delta", then that will disable the memory intensive deltra compression plumbing for those special cases where people need to track gzip archives (similarly another line "*.bz2 -delta" for bzip2 archives and so on). Since these files can't supposedly be compressed much more, I think Git ought to have a default heuristic to not attempt any compression on them.
Cheers
Alif