git status --> Out of memory, realloc failed

Dear Git group,

I use Git at a web hosting service, where my user account has a memory limit of 768 MB:

(uiserver):p7715773:~$ uname -a Linux infongp-de15 3.14.0-ui16322-uiabi1-infong-amd64 #1 SMP Debian 3.14.79-2~ui80+4 (2016-11-17) x86_64 GNU/Linux

(uiserver):p7715773:~$ git --version git version 2.1.4

The problem is that `git status` fails with an out of memory error:

(uiserver):p7715773:~/cafu$ git status fatal: Out of memory, realloc failed fatal: recursion detected in die handler

I talked to their support and their suggestion was to set a couple of memory constraints and to run `git gc`. This seemed to work well – but `git status` still fails:

(uiserver):p7715773:~/cafu$ cat ~/.gitconfig
[color]
     ui = auto
[user]
     name = Carsten Fuchs
     email = carsten.fuchs@cafu.de
[core]
     editor = nano
     pager = less -M -FRXS
     packedgitwindowsize = 30m
     packedgitlimit = 40m
[i18n]
     commitEncoding = ISO-8859-1
[pack]
     threads = 1
     windowMemory = 10m
     packSizeLimit = 20m
     deltaCacheSize = 30m

(uiserver):p7715773:~/cafu$ git gc Zähle Objekte: 44293, Fertig. Komprimiere Objekte: 100% (24534/24534), Fertig. Schreibe Objekte: 100% (44293/44293), Fertig. Total 44293 (delta 17560), reused 41828 (delta 16708)

(uiserver):p7715773:~/cafu$ git status fatal: Out of memory, realloc failed fatal: recursion detected in die handler

The repository is tracking about 19000 files which together take 260 MB. The git server version is 2.7.4.1.g5468f9e (Bitbucket)

Well, their next response was that they have no solution for me – except, unsurprisingly, coaxing me into a more expensive hosting package.

I've read the Git man page about `git config`, but was not able to come up with anything to improve the situation.

Any ideas what I could do to reduce the memory consumption of `git status`?

Best regards, Carsten

PS: Many thanks to Philip Oakley for initial advice at git-users, to 
which I should have properly subscribed in the first place, as the 
Google Groups interface seems to lose messages (mine at least, and 
inadvertently posts them as HTML) and gmane's NNTP interface reports 
that it is unidirectional/read-only.

Am 25.02.2017 um 11:13 schrieb Carsten Fuchs:

What's the output of "ulimit -a"?

That's quite old. Can you try a more recent version easily (2.12.0 just came out)? I don't remember a specific fix, and memory usage perhaps even increased with newer versions, but ruling out already fixed issues would be nice.

Is your repository publicly accessible?

René

Hi René,

Am 01.03.2017 um 11:02 schrieb René Scharfe:

(uiserver):p7715773:~$ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 1 file size (blocks, -f) unlimited pending signals (-i) 16382 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 512 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) 1800 max user processes (-u) 42 virtual memory (kbytes, -v) 786432 file locks (-x) unlimited

I don't think that they have any newer version available, having just upgraded from Git 1.7 a couple of weeks ago... (1und1)

Git 1.7 used to work for me in the same environment, but iirc they also said they switched from 32-bit to 64-bit edition and blame the increased memory consumption on that change.

I'll ask for a newer version, but I'd be surprised if this happened.

Unfortunately, no. There are no big secrets in there, but just a couple of database details so that I cannot make it universally available. I can gladly give you access though. (E.g. by adding your public SSH key?)

Best regards, Carsten

On Thu, Mar 2, 2017 at 3:12 AM, Carsten Fuchs <carsten.fuchs@cafu.de> wrote:

You could also try "git fast-export --anonymize" (read the doc first). I'm not sure if it keeps blob size though, and delta chain might also matter so an anonymized repo may or may not share the same problem, I'm not sure.

Hi Duy,

Am 2017-03-02 um 10:31 schrieb Duy Nguyen:

Eventually this was not needed, but thanks for letting me know about it! I've not been aware of this feature beforehand, learned something.

Best regards, Carsten

Am 01.03.2017 um 21:12 schrieb Carsten Fuchs:

When I use ulimit -v with lower and lower numbers I can provoke mmap failures on bigger pack files, but not the realloc failures that you're seeing. And your packs should be only up to 20MB anyway (you can check that with "ls -l .git/objects/pack/*.pack").

I'd rather not look at semi-confidential data, and you probably shouldn't hand it to a stranger on the internet anyway. ;)

So a shot in the dark: Do you have a lot of untracked files? You could check by cloning your repository locally (which copies only tracked contents). Does "git status" work on the clone?

Another one, darker yet: Does "git config core.preloadIndex 0" help?

René

Hi René,

Am 02.03.2017 um 21:04 schrieb René Scharfe:

Yes, there are 9 of them, all about 20 MB in size.

I had about 40 modified or untracked files with a combined file size of about 5 MB.

Before I got your latest mail, I tried something else: Instead of connecting to the system with the repository via SSH as usual, I accessed the repository by mounting its file system via gvfs-mount onto my local desktop machine. Then I used my local, normally working Git to commit all modified and some untracked files (and deleted those that were left). Running `git status` and other commands took a very long time over a network connection of only 6 MBit/s, but eventually I got everything committed and pushed.

Quite in the spirit of your above words, getting rid of untracked files helped: Logging normally via SSH into the remote machine again, `git status` ran normally.

Only then did I see your mail and got another clone, where `git status` worked equally well – but then `git diff` failed in a similar manner, continuing the problems of `git status`:

(uiserver):p7715773:~/__TEST__$ git clone https://CarstenF@bitbucket.org/CarstenF/website-cafu.git Cloning into 'website-cafu'... Password for 'https://CarstenF@bitbucket.org': remote: Counting objects: 44359, done. remote: Compressing objects: 100% (28777/28777), done. remote: Total 44359 (delta 19201), reused 38661 (delta 14143) Receiving objects: 100% (44359/44359), 168.28 MiB | 20.95 MiB/s, done. Resolving deltas: 100% (19201/19201), done. Checking connectivity... done. Checking out files: 100% (18524/18524), done.

(uiserver):p7715773:~/__TEST__$ cd website-cafu/

(uiserver):p7715773:~/__TEST__/website-cafu$ git status On branch master Your branch is up-to-date with 'origin/master'. nothing to commit, working directory clean

(uiserver):p7715773:~/__TEST__/website-cafu$ git diff fatal: unable to create threaded lstat

(uiserver):p7715773:~/__TEST__/website-cafu$ git diff fatal: Out of memory, realloc failed fatal: Out of memory, realloc failed fatal: Out of memory, realloc failed fatal: recursion detected in die handler

That's it!  :-)

Once that was set, both `git status` and `git diff` worked normally again!

I cannot test `git status` right now in the same situation as before, as I no longer have a working directory that is "as dirty" as before, but the above setting seems to fix the problem promptly and thoroughly!

Many thanks for your help!   :-)

Best regards, Carsten