{"thread":{"id":"54187","subject":"possible segfault wint index threading","startedAt":"2020-09-04T14:10:05Z","lastAt":"2020-09-04T19:28:31Z","messageCount":3,"participants":["Sandor Bodo-Merle","René Scharfe","Elijah Newren"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"405040","messageId":"CABLWAfQhWUSUS8p2XoGcNMUAp-vatD+-pDMngiJJW5qHG2CYCQ@mail.gmail.com","threadId":"54187","inReplyTo":null,"subject":"possible segfault wint index threading","fromName":"Sandor Bodo-Merle","fromEmail":"sbodomerle@gmail.com","sentAt":"2020-09-04T14:08:08Z","receivedAt":"2020-09-04T14:10:05Z","isPatch":false,"sender":{"key":"sbodomerle@gmail.com","avatar":null},"body":"Since commit 44c7e1a7e0 (mem-pool: use more standard initialization\nand finalization, 2020-08-15) i get a SIGSEGV in a linux kernel clone\nwith several remotes added:\n\n(gdb) set args \"fetch\" \"stable\"\n(gdb) run\nStarting program: /home/sbodo/usr/bin/git \"fetch\" \"stable\"\n[Thread debugging using libthread_db enabled]\nUsing host libthread_db library \"/lib/x86_64-linux-gnu/libthread_db.so.1\".\n[New Thread 0x7ffff6927700 (LWP 14810)]\n\nThread 1 \"git\" received signal SIGSEGV, Segmentation fault.\nmem_pool_init (pool=0x0, initial_size=initial_size@entry=7031819) at\nmem-pool.c:40\n40 memset(pool, 0, sizeof(*pool));\n(gdb) bt\n#0 mem_pool_init (pool=0x0, initial_size=initial_size@entry=7031819)\nat mem-pool.c:40\n#1 0x00005555556eb954 in load_cache_entries_threaded\n(ieot=0x555555a9f440, nr_threads=, mmap_size=6375563,\nmmap=0x7ffff6928000 \"DIRC\", istate=0x555555a981e0 ) at\nread-cache.c:2109\n#2 do_read_index (istate=istate@entry=0x555555a981e0 ,\npath=path@entry=0x555555a9ed50 \".git/index\",\nmust_exist=must_exist@entry=0) at read-cache.c:2229\n#3 0x00005555556ef68d in do_read_index (must_exist=0,\npath=0x555555a9ed50 \".git/index\", istate=0x555555a981e0 )\nat cache.h:1152\n#4 read_index_from (istate=0x555555a981e0 ,\npath=0x555555a9ed50 \".git/index\", gitdir=0x555555a9cba0 \".git\") at\nread-cache.c:2296\n#5 0x000055555573e4b0 in get_oid_with_context_1\n(repo=repo@entry=0x555555a84dc0 ,\nname=name@entry=0x5555557e8a92 \":.gitmodules\", flags=flags@entry=0,\nprefix=prefix@entry=0x0,\n oid=oid@entry=0x7fffffffd890, oc=oc@entry=0x7fffffffd800) at\nsha1-name.c:1843\n#6 0x000055555573eee3 in get_oid_with_context (oc=0x7fffffffd800,\noid=0x7fffffffd890, flags=0, str=0x5555557e8a92 \":.gitmodules\",\nrepo=0x555555a84dc0 ) at sha1-name.c:1940\n#7 repo_get_oid (r=r@entry=0x555555a84dc0 ,\nname=name@entry=0x5555557e8a92 \":.gitmodules\",\noid=oid@entry=0x7fffffffd890) at sha1-name.c:1596\n#8 0x0000555555748a0f in config_from_gitmodules\n(fn=fn@entry=0x555555748b30 ,\nrepo=0x555555a84dc0 , data=data@entry=0x7fffffffd8f0) at\nsubmodule-config.c:650\n#9 0x0000555555749cda in config_from_gitmodules (data=0x7fffffffd8f0,\nrepo=, fn=0x555555748b30 ) at\nsubmodule-config.c:638\n#10 fetch_config_from_gitmodules\n(max_children=max_children@entry=0x555555a47994\n,\nrecurse_submodules=recurse_submodules@entry=0x555555a4798c\n)\n at submodule-config.c:796\n#11 0x00005555555a92bb in cmd_fetch (argc=2, argv=0x7fffffffde20,\nprefix=0x0) at builtin/fetch.c:1774\n#12 0x000055555557232e in run_builtin (argv=,\nargc=, p=) at git.c:450\n#13 handle_builtin (argc=, argv=) at git.c:675\n#14 0x00005555555734a5 in run_argv (argv=0x7fffffffdb80,\nargcp=0x7fffffffdb8c) at git.c:742\n#15 cmd_main (argc=, argv=) at git.c:873\n#16 0x0000555555571f48 in main (argc=3, argv=0x7fffffffde18) at common-main.c:52\n\n\n[System Info]\ngit version:\ngit version 2.28.0.394.ge197136389\ncpu: x86_64\nbuilt from commit: e19713638985533ce461db072b49112da5bd2042\nsizeof-long: 8\nsizeof-size_t: 8\nshell-path: /bin/sh\nuname: Linux 5.4.0-42-generic #46~18.04.1-Ubuntu SMP Fri Jul 10\n07:21:24 UTC 2020 x86_64\ncompiler info: gnuc: 7.5\nlibc info: glibc: 2.27\n$SHELL (typically, interactive shell): /bin/bash\n\n\n[Enabled Hooks]\npost-commit\npost-checkout\npost-merge\npost-rewrite\n\nIt appears that i have \"index.threads\" set to \"true\" in my config.\nOnce i set it to false the segfault goes away.\n\nbr\n\nSandor\n"},{"id":"405054","messageId":"dbccbf13-9ebb-2464-4325-cac4c7767f74@web.de","threadId":"54187","inReplyTo":"CABLWAfQhWUSUS8p2XoGcNMUAp-vatD+-pDMngiJJW5qHG2CYCQ@mail.gmail.com","subject":"[PATCH] read-cache: fix mem-pool allocation for multi-threaded index loading","fromName":"René Scharfe","fromEmail":"l.s.r@web.de","sentAt":"2020-09-04T17:33:55Z","receivedAt":"2020-09-04T18:10:30Z","isPatch":true,"sender":{"key":"l.s.r@web.de","avatar":"https://avatars.githubusercontent.com/u/26122331?v=4"},"body":"44c7e1a7e0 (mem-pool: use more standard initialization and finalization,\n2020-08-15) moved the allocation of the mem-pool structure to callers.\nIt also added an allocation to load_cache_entries_threaded(), but for an\nunrelated mem-pool. Fix that by allocating the correct one instead --\nthe one that is initialized two lines later.\n\nReported-by: Sandor Bodo-Merle \nSigned-off-by: René Scharfe \n---\n\"GIT_TEST_INDEX_THREADS=3 make test\" passes with this patch, but fails a\nlot without it.\n\n read-cache.c | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex fa291cdbee..ecf6f68994 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -2101,7 +2101,7 @@ static unsigned long load_cache_entries_threaded(struct index_state *istate, con\n \t\tnr = 0;\n \t\tfor (j = p->ieot_start; j < p->ieot_start + p->ieot_blocks; j++)\n \t\t\tnr += p->ieot->entries[j].nr;\n-\t\tistate->ce_mem_pool = xmalloc(sizeof(*istate->ce_mem_pool));\n+\t\tp->ce_mem_pool = xmalloc(sizeof(*istate->ce_mem_pool));\n \t\tif (istate->version == 4) {\n \t\t\tmem_pool_init(p->ce_mem_pool,\n \t\t\t\testimate_cache_size_from_compressed(nr));\n--\n2.28.0\n\n"},{"id":"405056","messageId":"CABPp-BGQA98SOr4-+8LKd-V=oqB98hoDGh+AsvOGHjsM2ko=iA@mail.gmail.com","threadId":"54187","inReplyTo":"dbccbf13-9ebb-2464-4325-cac4c7767f74@web.de","subject":"Re: [PATCH] read-cache: fix mem-pool allocation for multi-threaded index loading","fromName":"Elijah Newren","fromEmail":"newren@gmail.com","sentAt":"2020-09-04T19:28:17Z","receivedAt":"2020-09-04T19:28:31Z","isPatch":true,"sender":{"key":"newren@gmail.com","avatar":"https://avatars.githubusercontent.com/u/5455730?v=4"},"body":"On Fri, Sep 4, 2020 at 10:34 AM René Scharfe wrote:\n>\n> 44c7e1a7e0 (mem-pool: use more standard initialization and finalization,\n> 2020-08-15) moved the allocation of the mem-pool structure to callers.\n> It also added an allocation to load_cache_entries_threaded(), but for an\n> unrelated mem-pool. Fix that by allocating the correct one instead --\n> the one that is initialized two lines later.\n\nWhoops; that's a somewhat embarrassing mistake. Thanks for fixing it up for me.\n\nReviewed-by: Elijah Newren \n\n>\n> Reported-by: Sandor Bodo-Merle \n> Signed-off-by: René Scharfe \n> ---\n> \"GIT_TEST_INDEX_THREADS=3 make test\" passes with this patch, but fails a\n> lot without it.\n>\n> read-cache.c | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/read-cache.c b/read-cache.c\n> index fa291cdbee..ecf6f68994 100644\n> --- a/read-cache.c\n> +++ b/read-cache.c\n> @@ -2101,7 +2101,7 @@ static unsigned long load_cache_entries_threaded(struct index_state *istate, con\n> nr = 0;\n> for (j = p->ieot_start; j < p->ieot_start + p->ieot_blocks; j++)\n> nr += p->ieot->entries[j].nr;\n> - istate->ce_mem_pool = xmalloc(sizeof(*istate->ce_mem_pool));\n> + p->ce_mem_pool = xmalloc(sizeof(*istate->ce_mem_pool));\n> if (istate->version == 4) {\n> mem_pool_init(p->ce_mem_pool,\n> estimate_cache_size_from_compressed(nr));\n> --\n> 2.28.0\n"}]}