{"thread":{"id":"53586","subject":"mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e","startedAt":"2020-05-31T10:52:48Z","lastAt":"2020-06-02T16:49:50Z","messageCount":6,"participants":["Jan Christoph Uhde","Jeff King","Eric Sunshine","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"398950","messageId":"54a3a798-0387-64df-be20-af69db124042@UhdeJc.com","threadId":"53586","inReplyTo":null,"subject":"mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e","fromName":"Jan Christoph Uhde","fromEmail":"jan@uhdejc.com","sentAt":"2020-05-31T10:46:30Z","receivedAt":"2020-05-31T10:52:48Z","isPatch":false,"sender":{"key":"jan@uhdejc.com","avatar":null},"body":"Hi,\n\nwhen calling `git diff --quiet` on the gcc git repository\nmy system fails reproducible with a failed mmap. I have\nnoticed this possible bug with git in debian unstable and\nthen build git from source. But I am not able to get a core\nor backtrace when gdb is attached.\n\nThe command I have used to build git is:\n\nmake clean; CFLAGS=\"-O0 -g -fsanitize=address\" DEBUG=1 make -j24\n\n\nMaybe you can give me some advice how to further debug this.\n\nJan\n\n--------------------------------------------------------------\n\nThe output below was created with the system git not\nthe self-compiled version (that behaves in the same way).\n\n» ulimit -a; git --version; git diff --quiet; git remote -v\ncore file size (blocks, -c) unlimited\ndata seg size (kbytes, -d) unlimited\nscheduling priority (-e) 0\nfile size (blocks, -f) unlimited\npending signals (-i) 514868\nmax locked memory (kbytes, -l) unlimited\nmax memory size (kbytes, -m) unlimited\nopen files (-n) 20000\npipe size (512 bytes, -p) 8\nPOSIX message queues (bytes, -q) 819200\nreal-time priority (-r) 0\nstack size (kbytes, -s) 8192\ncpu time (seconds, -t) unlimited\nmax user processes (-u) 2000\nvirtual memory (kbytes, -v) unlimited\nfile locks (-x) unlimited\ngit version 2.27.0.rc2\nfatal: mmap failed: Cannot allocate memory\norigin git://gcc.gnu.org/git/gcc.git (fetch)\norigin git://gcc.gnu.org/git/gcc.git (push)\n\n--------------------------------------------------------------\n\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b19717a.d\", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b19717a.d\", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b19717a.d\", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b19717a.d\", {st_mode=S_IFREG|0644, st_size=494, ...}) = 0\nopenat(AT_FDCWD, \"gcc/testsuite/gdc.test/fail_compilation/b19717a.d\", O_RDONLY) = 3\nmmap(NULL, 494, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f4a81a58000\nclose(3) = 0\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b3841.d\", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b3841.d\", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b3841.d\", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b3841.d\", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0\nlstat(\"gcc/testsuite/gdc.test/fail_compilation/b3841.d\", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0\nopenat(AT_FDCWD, \"gcc/testsuite/gdc.test/fail_compilation/b3841.d\", O_RDONLY) = 3\nmmap(NULL, 2643, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)\nopenat(AT_FDCWD, \"/usr/share/locale/en_US/LC_MESSAGES/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)\nopenat(AT_FDCWD, \"/usr/share/locale/en/LC_MESSAGES/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)\nwrite(2, \"fatal: mmap failed: Cannot alloc\"..., 43fatal: mmap failed: Cannot allocate memory\n) = 43\nexit_group(128) = ?\n+++ exited with 128 +++\n\n--------------------------------------------------------------\n\n» uname -a; free -m\nLinux edward 5.6.0-2-amd64 #1 SMP Debian 5.6.14-1 (2020-05-23) x86_64 GNU/Linux\n total used free shared buff/cache available\nMem: 128842 6279 86976 1084 35586 120439\nSwap: 122067 0 122067\n\n\n\n\nreading file \"/opt/gcc_repo/heaptrack.git.1558791.gz\" - please wait, this might take some time...\nDebuggee command was: git diff --quiet\nfinished reading file, now analyzing data:\n\nMOST CALLS TO ALLOCATION FUNCTIONS\n729067 calls to allocation functions with 484.95MB peak consumption from\n0x55ed4a44de70\n in /usr/bin/git\n66811 calls with 310.23MB peak consumption from:\n 0x55ed4a44dcdc\n in /usr/bin/git\n 0x55ed4a3b122e\n in /usr/bin/git\n 0x55ed4a3acc72\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n66811 calls with 0B peak consumption from:\n 0x55ed4a44dcdc\n in /usr/bin/git\n 0x55ed4a3aaf4c\n in /usr/bin/git\n 0x55ed4a3acc4c\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n66811 calls with 2.54MB peak consumption from:\n 0x55ed4a3accc0\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n65357 calls with 1.57MB peak consumption from:\n 0x55ed4a351cce\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n65357 calls with 8.61MB peak consumption from:\n 0x55ed4a34ffad\n in /usr/bin/git\n 0x55ed4a351c50\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 397920 from 210 other places\n\n152095 calls to allocation functions with 7.16KB peak consumption from\ninflateInit2_\n in /lib/x86_64-linux-gnu/libz.so.1\n66811 calls with 0B peak consumption from:\n 0x55ed4a458b9c\n in /usr/bin/git\n 0x55ed4a3aaf93\n in /usr/bin/git\n 0x55ed4a3acc4c\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n50269 calls with 7.16KB peak consumption from:\n 0x55ed4a458b9c\n in /usr/bin/git\n 0x55ed4a3aaf93\n in /usr/bin/git\n 0x55ed4a3ad084\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n35015 calls with 0B peak consumption from:\n 0x55ed4a458b9c\n in /usr/bin/git\n 0x55ed4a3abe9d\n in /usr/bin/git\n 0x55ed4a3ad369\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502b2\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n22747 calls to allocation functions with 0B peak consumption from\ninflate\n in /lib/x86_64-linux-gnu/libz.so.1\n22747 calls with 0B peak consumption from:\n 0x55ed4a458ebc\n in /usr/bin/git\n 0x55ed4a3abeac\n in /usr/bin/git\n 0x55ed4a3ad369\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502b2\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n668 calls to allocation functions with 3.13KB peak consumption from\n__strdup\n in /lib/x86_64-linux-gnu/libc.so.6\n573 calls with 75B peak consumption from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a30ff1c\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35010c\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n29 calls with 504B peak consumption from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a330abb\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a3330bf\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n29 calls with 2.12KB peak consumption from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a3309a5\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a3330bf\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n8 calls with 109B peak consumption from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a3309a5\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a333219\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n8 calls with 154B peak consumption from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a330abb\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a333219\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 21 from 19 other places\n\n630 calls to allocation functions with 13.37KB peak consumption from\n0x7f5839fd679e\n in /lib/x86_64-linux-gnu/libc.so.6\n573 calls with 0B peak consumption from:\n 0x55ed4a4604fb\n in /usr/bin/git\n 0x55ed4a44e53e\n in /usr/bin/git\n 0x55ed4a30fa95\n in /usr/bin/git\n 0x55ed4a30fca2\n in /usr/bin/git\n 0x55ed4a30feda\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35010c\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n12 calls with 3.73KB peak consumption from:\n 0x7f5839fd4bc4\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd3dba\n in /lib/x86_64-linux-gnu/libc.so.6\n setlocale\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a371396\n in /usr/bin/git\n 0x55ed4a24afbf\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n12 calls with 3.73KB peak consumption from:\n 0x7f5839fd4bc4\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd3dba\n in /lib/x86_64-linux-gnu/libc.so.6\n setlocale\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3713a7\n in /usr/bin/git\n 0x55ed4a24afbf\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n2 calls with 0B peak consumption from:\n 0x7f5839fd83c2\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd7cdf\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3bd6da\n in /usr/bin/git\n 0x55ed4a3b827c\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n2 calls with 2.05KB peak consumption from:\n 0x7f5839fda263\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd8497\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd7cdf\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3bd6da\n in /usr/bin/git\n 0x55ed4a3b827c\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 29 from 29 other places\n\n20 calls to allocation functions with 1.28KB peak consumption from\n0x7f583a275661\n in /lib64/ld-linux-x86-64.so.2\n20 calls with 1.28KB peak consumption from:\n _dl_allocate_tls\n in /lib64/ld-linux-x86-64.so.2\n pthread_create@@GLIBC_2.2.5\n in /lib/x86_64-linux-gnu/libpthread.so.0\n 0x55ed4a3b8240\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n7 calls to allocation functions with 0B peak consumption from\n_IO_file_doallocate\n in /lib/x86_64-linux-gnu/libc.so.6\n1 calls with 0B peak consumption from:\n _IO_doallocbuf\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_file_underflow\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_default_uflow\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a32dd71\n in /usr/bin/git\n 0x55ed4a32ef94\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a3330bf\n in /usr/bin/git\n 0x55ed4a333523\n in /usr/bin/git\n 0x55ed4a429fb1\n in /usr/bin/git\n 0x55ed4a24afda\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 calls with 0B peak consumption from:\n _IO_doallocbuf\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_file_underflow\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_default_uflow\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a32dd71\n in /usr/bin/git\n 0x55ed4a32ef94\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a40b9c2\n in /usr/bin/git\n 0x55ed4a40bb8c\n in /usr/bin/git\n 0x55ed4a40d14c\n in /usr/bin/git\n 0x55ed4a277c94\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 calls with 0B peak consumption from:\n _IO_doallocbuf\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_file_underflow\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_default_uflow\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a32dd71\n in /usr/bin/git\n 0x55ed4a32ef94\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a3330bf\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 calls with 0B peak consumption from:\n _IO_doallocbuf\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_file_underflow\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_default_uflow\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a32dd71\n in /usr/bin/git\n 0x55ed4a32ef94\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a333219\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 calls with 0B peak consumption from:\n _IO_doallocbuf\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_file_underflow\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_default_uflow\n in /lib/x86_64-linux-gnu/libc.so.6\n _IO_getline_info\n in /lib/x86_64-linux-gnu/libc.so.6\n fgets_unlocked\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd9d57\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fda263\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd8497\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd7cdf\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3bd6da\n in /usr/bin/git\n 0x55ed4a3b827c\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 2 from 2 other places\n\n3 calls to allocation functions with 72B peak consumption from\ntsearch\n in /lib/x86_64-linux-gnu/libc.so.6\n1 calls with 24B peak consumption from:\n 0x7f5839fdf96d\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a36605a\n in /usr/bin/git\n 0x55ed4a24c4e8\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 calls with 24B peak consumption from:\n 0x7f5839fdf96d\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a366091\n in /usr/bin/git\n 0x55ed4a24c4e8\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 calls with 24B peak consumption from:\n 0x7f5839fdf96d\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a40d2f2\n in /usr/bin/git\n 0x55ed4a277c94\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n1 calls to allocation functions with 24B peak consumption from\npcre2_general_context_create_8\n in /lib/x86_64-linux-gnu/libpcre2-8.so.0\n1 calls with 24B peak consumption from:\n 0x55ed4a377ba4\n in /usr/bin/git\n 0x55ed4a3eefe7\n in /usr/bin/git\n 0x55ed4a277d92\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n1 calls to allocation functions with 72.70KB peak consumption from\n0x7f5839e53565\n in /lib/x86_64-linux-gnu/libstdc++.so.6\n1 calls with 72.70KB peak consumption from:\n 0x7f583a273369\n in /lib64/ld-linux-x86-64.so.2\n 0x7f583a273468\n in /lib64/ld-linux-x86-64.so.2\n 0x7f583a2650c9\n in /lib64/ld-linux-x86-64.so.2\n\n\nPEAK MEMORY CONSUMERS\n484.95MB peak memory consumed over 729067 calls from\n0x55ed4a44de70\n in /usr/bin/git\n310.23MB consumed over 66811 calls from:\n 0x55ed4a44dcdc\n in /usr/bin/git\n 0x55ed4a3b122e\n in /usr/bin/git\n 0x55ed4a3acc72\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n104.67MB consumed over 50269 calls from:\n 0x55ed4a44dcdc\n in /usr/bin/git\n 0x55ed4a3aaf4c\n in /usr/bin/git\n 0x55ed4a3ad084\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n30.16MB consumed over 4672 calls from:\n 0x55ed4a44dcdc\n in /usr/bin/git\n 0x55ed4a44ddef\n in /usr/bin/git\n 0x55ed4a3ad13c\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n14.58MB consumed over 1 calls from:\n 0x55ed4a467ff6\n in /usr/bin/git\n 0x55ed4a4680a8\n in /usr/bin/git\n 0x55ed4a3c3662\n in /usr/bin/git\n 0x55ed4a3c78ac\n in /usr/bin/git\n 0x55ed4a3b83e7\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n8.61MB consumed over 65357 calls from:\n 0x55ed4a34ffad\n in /usr/bin/git\n 0x55ed4a351c50\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 16.68MB from 210 other places\n\n72.70KB peak memory consumed over 1 calls from\n0x7f5839e53565\n in /lib/x86_64-linux-gnu/libstdc++.so.6\n72.70KB consumed over 1 calls from:\n 0x7f583a273369\n in /lib64/ld-linux-x86-64.so.2\n 0x7f583a273468\n in /lib64/ld-linux-x86-64.so.2\n 0x7f583a2650c9\n in /lib64/ld-linux-x86-64.so.2\n\n13.37KB peak memory consumed over 630 calls from\n0x7f5839fd679e\n in /lib/x86_64-linux-gnu/libc.so.6\n3.73KB consumed over 12 calls from:\n 0x7f5839fd4bc4\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd3dba\n in /lib/x86_64-linux-gnu/libc.so.6\n setlocale\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a371396\n in /usr/bin/git\n 0x55ed4a24afbf\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n3.73KB consumed over 12 calls from:\n 0x7f5839fd4bc4\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd3dba\n in /lib/x86_64-linux-gnu/libc.so.6\n setlocale\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3713a7\n in /usr/bin/git\n 0x55ed4a24afbf\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n2.05KB consumed over 2 calls from:\n 0x7f5839fda263\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd8497\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd7cdf\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3bd6da\n in /usr/bin/git\n 0x55ed4a3b827c\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1.60KB consumed over 1 calls from:\n 0x7f5839fda263\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd8497\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd7cdf\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3bd6da\n in /usr/bin/git\n 0x55ed4a3b827c\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1.18KB consumed over 1 calls from:\n 0x55ed4a40d2f2\n in /usr/bin/git\n 0x55ed4a277c94\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 1.09KB from 29 other places\n\n7.16KB peak memory consumed over 152095 calls from\ninflateInit2_\n in /lib/x86_64-linux-gnu/libz.so.1\n7.16KB consumed over 50269 calls from:\n 0x55ed4a458b9c\n in /usr/bin/git\n 0x55ed4a3aaf93\n in /usr/bin/git\n 0x55ed4a3ad084\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n3.13KB peak memory consumed over 668 calls from\n__strdup\n in /lib/x86_64-linux-gnu/libc.so.6\n2.12KB consumed over 29 calls from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a3309a5\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a3330bf\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n504B consumed over 29 calls from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a330abb\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a3330bf\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n154B consumed over 8 calls from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a330abb\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a333219\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n109B consumed over 8 calls from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a3309a5\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a333219\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n75B consumed over 573 calls from:\n 0x55ed4a44dd38\n in /usr/bin/git\n 0x55ed4a30ff1c\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35010c\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 168B from 19 other places\n\n1.28KB peak memory consumed over 20 calls from\n0x7f583a275661\n in /lib64/ld-linux-x86-64.so.2\n1.28KB consumed over 20 calls from:\n _dl_allocate_tls\n in /lib64/ld-linux-x86-64.so.2\n pthread_create@@GLIBC_2.2.5\n in /lib/x86_64-linux-gnu/libpthread.so.0\n 0x55ed4a3b8240\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n72B peak memory consumed over 3 calls from\ntsearch\n in /lib/x86_64-linux-gnu/libc.so.6\n24B consumed over 1 calls from:\n 0x7f5839fdf96d\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a36605a\n in /usr/bin/git\n 0x55ed4a24c4e8\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n24B consumed over 1 calls from:\n 0x7f5839fdf96d\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a366091\n in /usr/bin/git\n 0x55ed4a24c4e8\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n24B consumed over 1 calls from:\n 0x7f5839fdf96d\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a40d2f2\n in /usr/bin/git\n 0x55ed4a277c94\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n24B peak memory consumed over 1 calls from\npcre2_general_context_create_8\n in /lib/x86_64-linux-gnu/libpcre2-8.so.0\n24B consumed over 1 calls from:\n 0x55ed4a377ba4\n in /usr/bin/git\n 0x55ed4a3eefe7\n in /usr/bin/git\n 0x55ed4a277d92\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n\nMOST TEMPORARY ALLOCATIONS\n261461 temporary allocations of 729067 allocations in total (35.86%) from\n0x55ed4a44de70\n in /usr/bin/git\n65356 temporary allocations of 65356 allocations in total (100.00%) from:\n 0x55ed4a41ac66\n in /usr/bin/git\n 0x55ed4a41b434\n in /usr/bin/git\n 0x55ed4a30fe5a\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35010c\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n65344 temporary allocations of 65344 allocations in total (100.00%) from:\n 0x55ed4a41ac66\n in /usr/bin/git\n 0x55ed4a41b434\n in /usr/bin/git\n 0x55ed4a30fe5a\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a350217\n in /usr/bin/git\n 0x55ed4a350b20\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n65344 temporary allocations of 65344 allocations in total (100.00%) from:\n 0x55ed4a41ac66\n in /usr/bin/git\n 0x55ed4a41b434\n in /usr/bin/git\n 0x55ed4a30fe5a\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35044a\n in /usr/bin/git\n 0x55ed4a350acc\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n65342 temporary allocations of 65357 allocations in total (99.98%) from:\n 0x55ed4a41ac66\n in /usr/bin/git\n 0x55ed4a41b434\n in /usr/bin/git\n 0x55ed4a30fe5a\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35010c\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n29 temporary allocations of 29 allocations in total (100.00%) from:\n 0x55ed4a44dcdc\n in /usr/bin/git\n 0x55ed4a3306fb\n in /usr/bin/git\n 0x55ed4a3308fd\n in /usr/bin/git\n 0x55ed4a330988\n in /usr/bin/git\n 0x55ed4a3350ee\n in /usr/bin/git\n 0x55ed4a32f251\n in /usr/bin/git\n 0x55ed4a3327ad\n in /usr/bin/git\n 0x55ed4a3330bf\n in /usr/bin/git\n 0x55ed4a333354\n in /usr/bin/git\n 0x55ed4a333e7b\n in /usr/bin/git\n 0x55ed4a277d73\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 46B from 210 other places\n\n129348 temporary allocations of 152095 allocations in total (85.04%) from\ninflateInit2_\n in /lib/x86_64-linux-gnu/libz.so.1\n66811 temporary allocations of 66811 allocations in total (100.00%) from:\n 0x55ed4a458b9c\n in /usr/bin/git\n 0x55ed4a3aaf93\n in /usr/bin/git\n 0x55ed4a3acc4c\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n50269 temporary allocations of 50269 allocations in total (100.00%) from:\n 0x55ed4a458b9c\n in /usr/bin/git\n 0x55ed4a3aaf93\n in /usr/bin/git\n 0x55ed4a3ad084\n in /usr/bin/git\n 0x55ed4a3ad395\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502d0\n in /usr/bin/git\n 0x55ed4a350b09\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n12268 temporary allocations of 35015 allocations in total (35.04%) from:\n 0x55ed4a458b9c\n in /usr/bin/git\n 0x55ed4a3abe9d\n in /usr/bin/git\n 0x55ed4a3ad369\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502b2\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n22747 temporary allocations of 22747 allocations in total (100.00%) from\ninflate\n in /lib/x86_64-linux-gnu/libz.so.1\n22747 temporary allocations of 22747 allocations in total (100.00%) from:\n 0x55ed4a458ebc\n in /usr/bin/git\n 0x55ed4a3abeac\n in /usr/bin/git\n 0x55ed4a3ad369\n in /usr/bin/git\n 0x55ed4a412692\n in /usr/bin/git\n 0x55ed4a41271a\n in /usr/bin/git\n 0x55ed4a3502b2\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n\n580 temporary allocations of 630 allocations in total (92.06%) from\n0x7f5839fd679e\n in /lib/x86_64-linux-gnu/libc.so.6\n572 temporary allocations of 573 allocations in total (99.83%) from:\n 0x55ed4a4604fb\n in /usr/bin/git\n 0x55ed4a44e53e\n in /usr/bin/git\n 0x55ed4a30fa95\n in /usr/bin/git\n 0x55ed4a30fca2\n in /usr/bin/git\n 0x55ed4a30feda\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35010c\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n2 temporary allocations of 2 allocations in total (100.00%) from:\n 0x7f5839fd83c2\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd7cdf\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3bd6da\n in /usr/bin/git\n 0x55ed4a3b827c\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 temporary allocations of 1 allocations in total (100.00%) from:\n 0x55ed4a4604fb\n in /usr/bin/git\n 0x55ed4a44e53e\n in /usr/bin/git\n 0x55ed4a30fa95\n in /usr/bin/git\n 0x55ed4a310474\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35010c\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 temporary allocations of 2 allocations in total (50.00%) from:\n 0x7f5839fda263\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd8497\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x7f5839fd7cdf\n in /lib/x86_64-linux-gnu/libc.so.6\n 0x55ed4a3bd6da\n in /usr/bin/git\n 0x55ed4a3b827c\n in /usr/bin/git\n 0x55ed4a3b83fc\n in /usr/bin/git\n 0x55ed4a278626\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n1 temporary allocations of 1 allocations in total (100.00%) from:\n 0x55ed4a4604fb\n in /usr/bin/git\n 0x55ed4a44e53e\n in /usr/bin/git\n 0x55ed4a30fa95\n in /usr/bin/git\n 0x55ed4a3103ec\n in /usr/bin/git\n 0x55ed4a310b28\n in /usr/bin/git\n 0x55ed4a338cc1\n in /usr/bin/git\n 0x55ed4a33ba3d\n in /usr/bin/git\n 0x55ed4a35010c\n in /usr/bin/git\n 0x55ed4a350ab7\n in /usr/bin/git\n 0x55ed4a351ede\n in /usr/bin/git\n 0x55ed4a345bae\n in /usr/bin/git\n 0x55ed4a27863b\n in /usr/bin/git\n 0x55ed4a24b464\n in /usr/bin/git\n 0x55ed4a24c523\n in /usr/bin/git\n 0x55ed4a24aff9\n in /usr/bin/git\n __libc_start_main\n in /lib/x86_64-linux-gnu/libc.so.6\n and 3B from 29 other places\n\n\ntotal runtime: 2.67s.\nbytes allocated in total (ignoring deallocations): 3.51GB (1.32GB/s)\ncalls to allocation functions: 905239 (339294/s)\ntemporary memory allocations: 414136 (155223/s)\npeak heap memory consumption: 485.04MB\npeak RSS (including heaptrack overhead): 11.14GB\ntotal memory leaked: 485.04MB\n"},{"id":"398970","messageId":"20200601044511.GA2529317@coredump.intra.peff.net","threadId":"53586","inReplyTo":"54a3a798-0387-64df-be20-af69db124042@UhdeJc.com","subject":"Re: mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-06-01T04:45:11Z","receivedAt":"2020-06-01T04:45:14Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Sun, May 31, 2020 at 12:46:30PM +0200, Jan Christoph Uhde wrote:\n\n> lstat(\"gcc/testsuite/gdc.test/fail_compilation/b3841.d\", {st_mode=S_IFREG|0644, st_size=2643, ...}) = 0\n> openat(AT_FDCWD, \"gcc/testsuite/gdc.test/fail_compilation/b3841.d\", O_RDONLY) = 3\n> mmap(NULL, 2643, PROT_READ, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)\n> openat(AT_FDCWD, \"/usr/share/locale/en_US/LC_MESSAGES/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)\n> openat(AT_FDCWD, \"/usr/share/locale/en/LC_MESSAGES/libc.mo\", O_RDONLY) = -1 ENOENT (No such file or directory)\n> write(2, \"fatal: mmap failed: Cannot alloc\"..., 43fatal: mmap failed: Cannot allocate memory\n> ) = 43\n\nSo we open a file in the working tree, and that mmap fails. I guess this\nis probably the call to xmmap() in diff_populate_filespec(). That file\nisn't particularly large.\n\nIs it possible that your local repository has large number of packs? Git\nwill leave open maps to each pack's index file, plus some packs\nthemselves (ones we're accessing, plus we map+close small ones), plus\nwhatever maps are used by libc to malloc. The kernel default limit for\nthe number of maps is 65530. If you have on the order of 30,000 packs\nyou might run into this limit.\n\nYou can check the number of packs with \"git count-objects -v\", and the\nmap limit with \"sysctl vm.max_map_count\".\n\nIf that's the problem, the solution is to repack (which should also\ngenerally improve performance). If you have trouble repacking due to the\nlimits, you can overcome the chicken and egg with:\n\n sysctl -w vm.max_map_count=131060\n\n-Peff\n"},{"id":"398982","messageId":"20200601165408.GA2536619@coredump.intra.peff.net","threadId":"53586","inReplyTo":"cfc79aec-ec85-dbec-e37b-6b7035b4c5a4@UhdeJc.com","subject":"Re: mmap failure in master 1aa69c73577df21f5e37e47cc40cf44fc049121e","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-06-01T16:54:08Z","receivedAt":"2020-06-01T16:54:10Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"[re-adding list to cc; I think there are some interesting bits for\ndiscussion here]\n\nOn Mon, Jun 01, 2020 at 08:41:08AM +0200, Jan Christoph Uhde wrote:\n\n> > Is it possible that your local repository has large number of packs? Git\n> > will leave open maps to each pack's index file, plus some packs\n> > themselves (ones we're accessing, plus we map+close small ones), plus\n> > whatever maps are used by libc to malloc. The kernel default limit for\n> > the number of maps is 65530. If you have on the order of 30,000 packs\n> > you might run into this limit.\n> > \n> > You can check the number of packs with \"git count-objects -v\", and the\n> \n> » git count-objects -v\n> count: 0\n> size: 0\n> in-pack: 2399361\n> packs: 1\n> size-pack: 919395\n> prune-packable: 0\n> garbage: 0\n> size-garbage: 0\n\nOK, so that's just one pack (with a few million objects, which is\nexpected).\n\n> > If that's the problem, the solution is to repack (which should also\n> > generally improve performance). If you have trouble repacking due to the\n> > limits, you can overcome the chicken and egg with:\n> > \n> > sysctl -w vm.max_map_count=131060\n> \n> This fixes the problem indeed!\n\nNow that surprises me. However, I can reproduce the issue with a fresh\nclone of gcc:\n\n $ git clone https://github.com/gcc-mirror/gcc\n $ cd gcc\n $ git diff --quiet ;# works!\n $ find . -type f | xargs touch ;# dirty the index entry for each file\n $ git diff --quiet\n fatal: mmap failed: Cannot allocate memory\n\nThere are ~100k files in that repo. If we mmap each one as part of\ndiff_populate_filespec(), we're going to end up with too many maps.\n\nSo we can easily reproduce this case without having to download the huge\ngcc repo:\n\n git init repo\n cd repo\n for i in $(seq 70); do\n mkdir $i\n for j in $(seq 1000); do\n echo \"foo\" >$i/$j\n done\n done\n git add .\n git commit -m 'add a bunch of files'\n\n git diff --quiet\n git ls-files | xargs touch\n git diff --quiet\n\n> What could be the reason that the problem only occurs with the `--quiet` flag\n> that I use in my prompt command, but not when using `git diff` without the flag.\n\nThat is curious, and I can reproduce it here.\n\nIn the non-quiet case, we queue each filepair that we find to be\nstat-dirty. And then we call diffcore_skip_stat_unmatch() from\ndiffcore_std() on the whole list. We load each file singly, see that\nit's not really a change (the index entries are just stat-dirty because\nof our touch), and then discard it.\n\nWhereas in the --quiet case, we hit this code in diff_change when\nqueuing each:\n\n if (options->flags.quick && options->skip_stat_unmatch &&\n !diff_filespec_check_stat_unmatch(options->repo, p))\n return;\n\n options->flags.has_changes = 1;\n\nThat code is trying to make sure we accurately set the has_changes flag\nin quick/quiet mode (because we can stop diffing as soon as we see a\nsingle change). Since none of these changes is interesting (they're all\njust stat-dirty entries), we have to keep going and look at each one.\nBut this code leaves the populated filespec in the queue. Doing this\nmakes \"diff --quiet\" succeed in this case:\n\ndiff --git a/diff.c b/diff.c\nindex d1ad6a3c4a..2535614735 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -6758,8 +6758,15 @@ void diff_change(struct diff_options *options,\n \t\treturn;\n \n \tif (options->flags.quick && options->skip_stat_unmatch &&\n-\t !diff_filespec_check_stat_unmatch(options->repo, p))\n+\t !diff_filespec_check_stat_unmatch(options->repo, p)) {\n+\t\t/*\n+\t\t * discard any populated data; this entry is uninteresting;\n+\t\t * we probably ought to avoid queuing the pair at all!\n+\t\t */\n+\t\tdiff_free_filespec_data(p->one);\n+\t\tdiff_free_filespec_data(p->two);\n \t\treturn;\n+\t}\n \n \toptions->flags.has_changes = 1;\n }\n\nBut that's only because these aren't \"real\" changes. If we swap out our\n\"touch\" for:\n\n git ls-files | while read fn; do\n echo bar >$fn\n done\n\nthen we have real changes. Our --quiet case will exit immediately once\nit sees a change, so it's OK. But now the non-quiet one will fail,\nbecause it's going to load each file to confirm that it's an actual\nchange, but leave the mmap in place for when we do the actual\ncontent-level diff.\n\nSo I think in general we do have problems diffing more than 65k working\ntree files in a single process because of this limit. It may still be\nworth doing something along the lines of the patch above, though,\nbecause it seems more likely for somebody to have 65k stat-dirty files\nthan 65k actual changes.\n\n-Peff\n"},{"id":"398994","messageId":"20200601202218.GA2763518@coredump.intra.peff.net","threadId":"53586","inReplyTo":"20200601165408.GA2536619@coredump.intra.peff.net","subject":"[PATCH] diff: discard blob data from stat-unmatched pairs","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2020-06-01T20:22:18Z","receivedAt":"2020-06-01T20:22:22Z","isPatch":true,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Mon, Jun 01, 2020 at 12:54:08PM -0400, Jeff King wrote:\n\n> > What could be the reason that the problem only occurs with the `--quiet` flag\n> > that I use in my prompt command, but not when using `git diff` without the flag.\n> \n> That is curious, and I can reproduce it here.\n\nThis turned out to be quite an interesting bug to look at. The fix is\nsimple, but I tried to summarize my explorations below.\n\nI'm still curious why all of your files were stat-dirty. :) As a\nworkaround until you have a version of Git with the fix, you can\nmanually refresh the index with:\n\n git update-index --refresh\n\nbefore running git-diff.\n\n-- >8 --\nSubject: [PATCH] diff: discard blob data from stat-unmatched pairs\n\nWhen performing a tree-level diff against the working tree, we may find\nthat our index stat information is dirty, so we queue a filepair to be\nexamined later. If the actual content hasn't changed, we call this a\nstat-unmatch; the stat information was out of date, but there's no\nactual diff. Normally diffcore_std() would detect and remove these\nidentical filepairs via diffcore_skip_stat_unmatch(). However, when\n\"--quiet\" is used, we want to stop the diff as soon as we see any\nchanges, so we check for stat-unmatches immediately in diff_change().\n\nThat check may require us to actually load the file contents into the\npair of diff_filespecs. If we find that the pair isn't a stat-unmatch,\nthen no big deal; we'd likely load the contents later anyway to generate\na patch, do rename detection, etc, so we want to hold on to it. But if\nit a stat-unmatch, then we have no more use for that data; the whole\npoint is that we're going discard the pair. However, we never free the\nallocated diff_filespec data.\n\nIn most cases, keeping that data isn't a problem. We don't expect a lot\nof stat-unmatch entries, and since we're using --quiet, we'd quit as\nsoon as we saw such a real change anyway. However, there are extreme\ncases where it makes a big difference:\n\n 1. We'd generally mmap() the working tree half of the pair. And since\n the OS may limit the total number of maps, we can run afoul of this\n in large repositories. E.g.:\n\n $ cd linux\n $ git ls-files | wc -l\n 67959\n $ sysctl vm.max_map_count\n vm.max_map_count = 65530\n $ git ls-files | xargs touch ;# everything is stat-dirty!\n $ git diff --quiet\n fatal: mmap failed: Cannot allocate memory\n\n It should be unusual to have so many files stat-dirty, but it's\n possible if you've just run a script like \"sed -i\" or similar.\n\n After this patch, the above correctly exits with code 0.\n\n 2. Even if you don't hit mmap limits, the index half of the pair will\n have been pulled from the object database into heap memory. Again\n in a clone of linux.git, running:\n\n $ git ls-files | head -n 10000 | xargs touch\n $ git diff --quiet\n\n peaks at 145MB heap before this patch, and 94MB after.\n\nThis patch solves the problem by freeing any diff_filespec data we\npicked up during the \"--quiet\" stat-unmatch check in diff_changes.\nNobody is going to need that data later, so there's no point holding on\nto it. There are a few things to note:\n\n - we could skip queueing the pair entirely, which could in theory save\n a little work. But there's not much to save, as we need a\n diff_filepair to feed to diff_filespec_check_stat_unmatch() anyway.\n And since we cache the result of the stat-unmatch checks, a later\n call to diffcore_skip_stat_unmatch() call will quickly skip over\n them. The diffcore code also counts up the number of stat-unmatched\n pairs as it removes them. It's doubtful any callers would care about\n that in combination with --quiet, but we'd have to reimplement the\n logic here to be on the safe side. So it's not really worth the\n trouble.\n\n - I didn't write a test, because we always produce the correct output\n unless we run up against system mmap limits, which are both\n unportable and expensive to test against. Measuring peak heap\n would be interesting, but our perf suite isn't yet capable of that.\n\n - note that diff without \"--quiet\" does not suffer from the same\n problem. In diffcore_skip_stat_unmatch(), we detect the stat-unmatch\n entries and drop them immediately, so we're not carrying their data\n around.\n\n - you _can_ still trigger the mmap limit problem if you truly have\n that many files with actual changes. But it's rather unlikely. The\n stat-unmatch check avoids loading the file contents if the sizes\n don't match, so you'd need a pretty trivial change in every single\n file. Likewise, inexact rename detection might load the data for\n many files all at once. But you'd need not just 64k changes, but\n that many deletions and additions. The most likely candidate is\n perhaps break-detection, which would load the data for all pairs and\n keep it around for the content-level diff. But again, you'd need 64k\n actually changed files in the first place.\n\n So it's still possible to trigger this case, but it seems like \"I\n accidentally made all my files stat-dirty\" is the most likely case\n in the real world.\n\nReported-by: Jan Christoph Uhde \nSigned-off-by: Jeff King \n---\n diff.c | 5 ++++-\n 1 file changed, 4 insertions(+), 1 deletion(-)\n\ndiff --git a/diff.c b/diff.c\nindex d1ad6a3c4a..4d46fab5d3 100644\n--- a/diff.c\n+++ b/diff.c\n@@ -6758,8 +6758,11 @@ void diff_change(struct diff_options *options,\n \t\treturn;\n \n \tif (options->flags.quick && options->skip_stat_unmatch &&\n-\t !diff_filespec_check_stat_unmatch(options->repo, p))\n+\t !diff_filespec_check_stat_unmatch(options->repo, p)) {\n+\t\tdiff_free_filespec_data(p->one);\n+\t\tdiff_free_filespec_data(p->two);\n \t\treturn;\n+\t}\n \n \toptions->flags.has_changes = 1;\n }\n-- \n2.27.0.rc2.689.g04a94059a7\n\n"},{"id":"398996","messageId":"CAPig+cRyrEHNxY+Y-_E6e4DZB-sAiVdSB2T53bKP6UJOx_DJDA@mail.gmail.com","threadId":"53586","inReplyTo":"20200601202218.GA2763518@coredump.intra.peff.net","subject":"Re: [PATCH] diff: discard blob data from stat-unmatched pairs","fromName":"Eric Sunshine","fromEmail":"sunshine@sunshineco.com","sentAt":"2020-06-01T20:53:50Z","receivedAt":"2020-06-01T20:54:05Z","isPatch":true,"sender":{"key":"sunshine@sunshineco.com","avatar":"https://avatars.githubusercontent.com/u/163641?v=4"},"body":"On Mon, Jun 1, 2020 at 4:22 PM Jeff King wrote:\n> Subject: [PATCH] diff: discard blob data from stat-unmatched pairs\n>\n> When performing a tree-level diff against the working tree, we may find\n> that our index stat information is dirty, so we queue a filepair to be\n> examined later. If the actual content hasn't changed, we call this a\n> stat-unmatch; the stat information was out of date, but there's no\n> actual diff. Normally diffcore_std() would detect and remove these\n> identical filepairs via diffcore_skip_stat_unmatch(). However, when\n> \"--quiet\" is used, we want to stop the diff as soon as we see any\n> changes, so we check for stat-unmatches immediately in diff_change().\n>\n> That check may require us to actually load the file contents into the\n> pair of diff_filespecs. If we find that the pair isn't a stat-unmatch,\n> then no big deal; we'd likely load the contents later anyway to generate\n> a patch, do rename detection, etc, so we want to hold on to it. But if\n> it a stat-unmatch, then we have no more use for that data; the whole\n\ns/it/& is/\n\n> point is that we're going discard the pair. However, we never free the\n> allocated diff_filespec data.\n>\n> In most cases, keeping that data isn't a problem. We don't expect a lot\n> of stat-unmatch entries, and since we're using --quiet, we'd quit as\n> soon as we saw such a real change anyway. However, there are extreme\n> cases where it makes a big difference:\n>\n> 1. We'd generally mmap() the working tree half of the pair. And since\n> the OS may limit the total number of maps, we can run afoul of this\n> in large repositories. E.g.:\n>\n> $ cd linux\n> $ git ls-files | wc -l\n> 67959\n> $ sysctl vm.max_map_count\n> vm.max_map_count = 65530\n> $ git ls-files | xargs touch ;# everything is stat-dirty!\n> $ git diff --quiet\n> fatal: mmap failed: Cannot allocate memory\n>\n> It should be unusual to have so many files stat-dirty, but it's\n> possible if you've just run a script like \"sed -i\" or similar.\n>\n> After this patch, the above correctly exits with code 0.\n>\n> 2. Even if you don't hit mmap limits, the index half of the pair will\n> have been pulled from the object database into heap memory. Again\n> in a clone of linux.git, running:\n>\n> $ git ls-files | head -n 10000 | xargs touch\n> $ git diff --quiet\n>\n> peaks at 145MB heap before this patch, and 94MB after.\n>\n> This patch solves the problem by freeing any diff_filespec data we\n> picked up during the \"--quiet\" stat-unmatch check in diff_changes.\n> Nobody is going to need that data later, so there's no point holding on\n> to it. There are a few things to note:\n>\n> - we could skip queueing the pair entirely, which could in theory save\n> a little work. But there's not much to save, as we need a\n> diff_filepair to feed to diff_filespec_check_stat_unmatch() anyway.\n> And since we cache the result of the stat-unmatch checks, a later\n> call to diffcore_skip_stat_unmatch() call will quickly skip over\n> them. The diffcore code also counts up the number of stat-unmatched\n> pairs as it removes them. It's doubtful any callers would care about\n> that in combination with --quiet, but we'd have to reimplement the\n> logic here to be on the safe side. So it's not really worth the\n> trouble.\n>\n> - I didn't write a test, because we always produce the correct output\n> unless we run up against system mmap limits, which are both\n> unportable and expensive to test against. Measuring peak heap\n> would be interesting, but our perf suite isn't yet capable of that.\n>\n> - note that diff without \"--quiet\" does not suffer from the same\n> problem. In diffcore_skip_stat_unmatch(), we detect the stat-unmatch\n> entries and drop them immediately, so we're not carrying their data\n> around.\n>\n> - you _can_ still trigger the mmap limit problem if you truly have\n> that many files with actual changes. But it's rather unlikely. The\n> stat-unmatch check avoids loading the file contents if the sizes\n> don't match, so you'd need a pretty trivial change in every single\n> file. Likewise, inexact rename detection might load the data for\n> many files all at once. But you'd need not just 64k changes, but\n> that many deletions and additions. The most likely candidate is\n> perhaps break-detection, which would load the data for all pairs and\n> keep it around for the content-level diff. But again, you'd need 64k\n> actually changed files in the first place.\n>\n> So it's still possible to trigger this case, but it seems like \"I\n> accidentally made all my files stat-dirty\" is the most likely case\n> in the real world.\n>\n> Reported-by: Jan Christoph Uhde \n> Signed-off-by: Jeff King \n"},{"id":"399033","messageId":"xmqqmu5ltntk.fsf@gitster.c.googlers.com","threadId":"53586","inReplyTo":"20200601202218.GA2763518@coredump.intra.peff.net","subject":"Re: [PATCH] diff: discard blob data from stat-unmatched pairs","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2020-06-02T16:49:43Z","receivedAt":"2020-06-02T16:49:50Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King writes:\n\n> Subject: [PATCH] diff: discard blob data from stat-unmatched pairs\n>\n> When performing a tree-level diff against the working tree, we may find\n> that our index stat information is dirty, so we queue a filepair to be\n> examined later. If the actual content hasn't changed, we call this a\n> stat-unmatch; the stat information was out of date, but there's no\n> actual diff. Normally diffcore_std() would detect and remove these\n> identical filepairs via diffcore_skip_stat_unmatch(). However, when\n> \"--quiet\" is used, we want to stop the diff as soon as we see any\n> changes, so we check for stat-unmatches immediately in diff_change().\n>\n> That check may require us to actually load the file contents into the\n> pair of diff_filespecs. If we find that the pair isn't a stat-unmatch,\n> then no big deal; we'd likely load the contents later anyway to generate\n> a patch, do rename detection, etc, so we want to hold on to it. But if\n> it a stat-unmatch, then we have no more use for that data; the whole\n> point is that we're going discard the pair. However, we never free the\n> allocated diff_filespec data.\n\nNicely spotted. So we can discard when quiet is in effect after\nthis check, which makes sense.\n\nAfter reading the initial analysis, I wondered if the fix we did in\nf34b205f (diff: do not quit early on stat-dirty files, 2014-01-25)\nwas suboptimal and we should have instead done the \"if QUICK, check\nif the pair is merely stat-unmatch\" in the loop(s) that call\ndiff_change(), hoping that it may have given us a better control\nover the lifetime of the filespecs in each diff_filepair, but I do\nnot think it would made much difference.\n\n> \tif (options->flags.quick && options->skip_stat_unmatch &&\n> -\t !diff_filespec_check_stat_unmatch(options->repo, p))\n> +\t !diff_filespec_check_stat_unmatch(options->repo, p)) {\n> +\t\tdiff_free_filespec_data(p->one);\n> +\t\tdiff_free_filespec_data(p->two);\n> \t\treturn;\n> +\t}\n\nThanks. Will queue (with that s/it/& is/ typofix mentioned\nelsewhere).\n\n"}]}