Re: Git server eats all memory

Nguyen Thai Ngoc Duy <pclouds@gmail.com> wrote:

Jared Hance <jaredhance@gmail.com> wrote:

Hello Jared and Nguyen,

Thank you Nguyen for your command. I can now reproduce the problem without needing the network. I have been following Jared lead today on a potential memory leak. Here is what I found out.

I downloaded the latest release of git 1.7.2.1 and compiled it with debugging support. I ran valgrind on the command and found two memory leaks. I put the output at the bottom of the e-mail as it's not very interesting. I patched one of the leak in pack_objects.c but got the same problem: over 4G of memory consumption for a 4G repository.

I've come to the conclusion that it's not a memory leak. 

This afternoon I put macro around the following functions: xmalloc xmallocz, xrealloc, xcalloc and xmmap. It reported the line of code and size passed in each functions. I then run the result through a script that totaled the amount used by each bit of code.

Here are the top 3 consumers:

| function | source                     | size in M |
|----------+----------------------------+-----------|
| xrealloc | builtin/pack-objects.c:690 |        86 |
| xmallocz | patch-delta.c:36           |       301 |
| xmmap    | sha1_file.c:772            |      4393 |

I expected the malloc to take 4G but was surprised it didn't. It seems to be mmap taking all the memory. I am not familiar with that function, it looks like it's mapping memory to a file... Is it reasonable to mmap so much memory?

Today I chatted with someone on freenode #git and he reported the same problem on his 2G repository, I am glad I am not the only one seeing this ;)

I tried reading the code but it's going over my head. I'll look at is some more next monday.

If anyone is familiar with the code source of git I would love to have some insight into this.

Take care,

Ivan Kanis

PS: output of valgrind --leak-check=full

65 bytes in 1 blocks are definitely lost in loss record 4 of 7
   at 0x4C2260E: malloc (vg_replace_malloc.c:207)
   by 0x4C22797: realloc (vg_replace_malloc.c:429)
   by 0x4C600D: xrealloc (wrapper.c:80)
   by 0x4B7939: strbuf_grow (strbuf.c:70)
   by 0x4B80BA: strbuf_addf (strbuf.c:201)
   by 0x4832EF: system_path (exec_cmd.c:37)
   by 0x483411: setup_path (exec_cmd.c:104)
   by 0x404AF2: main (git.c:536)

512 bytes in 1 blocks are definitely lost in loss record 5 of 8
   at 0x4C203E4: calloc (vg_replace_malloc.c:397)
   by 0x4C5F9D: xcalloc (wrapper.c:96)
   by 0x445741: cmd_pack_objects (pack-objects.c:2117)
   by 0x4048EE: handle_internal_command (git.c:270)
   by 0x404B03: main (git.c:470)