Re: git status OOM on mmap of large file

From: brian m. carlson <sandals@crustytoothpaste.net>
Date: Jan 24, 2019, 00:39 UTC
Message-ID: <20190124003948.GS423984@genre.crustytoothpaste.net>
In-Reply-To: <20190122220714.GA6176@kitenet.net>

On Tue, Jan 22, 2019 at 06:07:14PM -0400, Joey Hess wrote:

Show 13 quoted lines

> joey@darkstar:~/tmp/t> ls -l big-file
> -rw-r--r-- 1 joey joey 11811160064 Jan 22 17:48 big-file
> joey@darkstar:~/tmp/t> git status
> fatal: Out of memory, realloc failed
> 
> This file is checked into git, but using a smudge/clean filter, so the actual
> data checked into git is a hash. I did so using git-annex v7 mode, but I
> suppose git lfs would cause the same problem.
> 
> [pid  6573] mmap(NULL, 11811164160, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
> 
> Why status needs to mmap a large file that is not modified
> and that is configured to pass through smudge/clean, I don't know.

I believe that currently, Git stores the smudge/clean output in memory until it writes it out. When using the persistent filter process, it's possible for the process to choose to abort the operation, so we store the data in memory until we get the status.

Theoretically, it should be possible for us to write this to a temporary file, and if necessary, rename into place, although I'm not sure how well that will work on Windows. File modes may also be tricky here. Patches are of course welcome.

-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204