Re: fatal: Out of memory, getdelim failed under NFS mounts

From: Junio C Hamano <gitster@pobox.com>
Date: Aug 10, 2017, 18:56 UTC
Message-ID: <xmqqzib72qvs.fsf@gitster.mtv.corp.google.com>
In-Reply-To: <8e307474-d180-6d98-2c6b-062f2181bd14@web.de>

René Scharfe <l.s.r@web.de> writes:

Show 16 quoted lines

> I doubt the type of file system matters.  The questions are: How much
> main memory do you have, what is git trying to cram into it, is there
> a way to reduce the memory footprint or do you need to add more RAM?
>
>> any recommendations on how to pin point the "offender"? ;)
> Running "GIT_TRACE=1 git pull --ff-only origin master" would be a
> good start, I think, to find out which of the different activities
> that pull is doing causes the out-of-memory error.
>
> "free" and "ulimit -a" can help you find out how much memory you can
> use.
>
> Also: What does "wc -L .git/FETCH_HEAD .git/packed-refs" report?
> getdelim() is used mostly to read lines from files like these and in
> the admittedly unlikely case that they are *really* long such an
> error would be expected.

There is only one getdelim() call, which was introduced in v2.5.0 timeframe, and it is used like this:

	r = getdelim(&sb->buf, &sb->alloc, term, fp);

	if (r > 0) {
		sb->len = r;
		return 0;
	}
	assert(r == -1);

	/*
	 * Normally we would have called xrealloc, which will try to free
	 * memory and recover. But we have no way to tell getdelim() to do so.
	 * Worse, we cannot try to recover ENOMEM ourselves, because we have
	 * no idea how many bytes were read by getdelim.
	 *
	 * Dying here is reasonable. It mirrors what xrealloc would do on
	 * catastrophic memory failure. We skip the opportunity to free pack
	 * memory and retry, but that's unlikely to help for a malloc small
	 * enough to hold a single line of input, anyway.
	 */
	if (errno == ENOMEM)
		die("Out of memory, getdelim failed");

So the function is returning -1 and leaving ENOMEM in errno on Yaroslav's system.

I wonder if we are truly hitting out of memory, though. The same symptom could bee seen if getdelim() does not touch errno when it returns -1, but some other system call earlier set it to ENOMEM, for example.

If the same version of Git is recompiled there without HAVE_GETDELIM defined, would it still die with out of memory (presumably inside the call to strbuf_grow() in the strbuf_getwholeline() function)?