{"thread":{"id":"45611","subject":"[PATCH v2 0/2] string-list: use ALLOC_GROW macro when reallocing","startedAt":"2017-04-05T21:09:41Z","lastAt":"2017-04-05T21:35:56Z","messageCount":4,"participants":["git@jeffhostetler.com","Jonathan Nieder"],"isPatch":true,"patchVersion":2,"patchTotal":2},"messages":[{"id":"316235","messageId":"20170405210920.56549-1-git@jeffhostetler.com","threadId":"45611","inReplyTo":null,"subject":"[PATCH v2 0/2] string-list: use ALLOC_GROW macro when reallocing","fromName":"","fromEmail":"git@jeffhostetler.com","sentAt":"2017-04-05T21:09:18Z","receivedAt":"2017-04-05T21:09:41Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nUse ALLOC_GROW() macro when reallocating a string_list array\nrather than simply increasing it by 32. This helps performance\nof status on very large repos on Windows.\n\nJeff Hostetler (2):\n string-list: use ALLOC_GROW macro when reallocing string_list\n p0005-status: time status on very large repo\n\n string-list.c | 5 +---\n t/perf/p0005-status.sh | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++\n 2 files changed, 71 insertions(+), 4 deletions(-)\n create mode 100644 t/perf/p0005-status.sh\n\n-- \n2.9.3\n\n"},{"id":"316236","messageId":"20170405210920.56549-2-git@jeffhostetler.com","threadId":"45611","inReplyTo":"20170405210920.56549-1-git@jeffhostetler.com","subject":"[PATCH v2 1/2] string-list: use ALLOC_GROW macro when reallocing string_list","fromName":"","fromEmail":"git@jeffhostetler.com","sentAt":"2017-04-05T21:09:19Z","receivedAt":"2017-04-05T21:09:42Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nUse ALLOC_GROW() macro when reallocing a string_list array\nrather than simply increasing it by 32. This is a performance\noptimization.\n\nDuring status on a very large repo and there are many changes,\na significant percentage of the total run time was spent\nreallocing the wt_status.changes array.\n\nThis change decreased the time in wt_status_collect_changes_worktree()\nfrom 125 seconds to 45 seconds on my very large repository.\n\nSigned-off-by: Jeff Hostetler \n---\n string-list.c | 5 +----\n 1 file changed, 1 insertion(+), 4 deletions(-)\n\ndiff --git a/string-list.c b/string-list.c\nindex 45016ad..003ca18 100644\n--- a/string-list.c\n+++ b/string-list.c\n@@ -41,10 +41,7 @@ static int add_entry(int insert_at, struct string_list *list, const char *string\n \tif (exact_match)\n \t\treturn -1 - index;\n \n-\tif (list->nr + 1 >= list->alloc) {\n-\t\tlist->alloc += 32;\n-\t\tREALLOC_ARRAY(list->items, list->alloc);\n-\t}\n+\tALLOC_GROW(list->items, list->nr+1, list->alloc);\n \tif (index < list->nr)\n \t\tmemmove(list->items + index + 1, list->items + index,\n \t\t\t\t(list->nr - index)\n-- \n2.9.3\n\n"},{"id":"316237","messageId":"20170405210920.56549-3-git@jeffhostetler.com","threadId":"45611","inReplyTo":"20170405210920.56549-1-git@jeffhostetler.com","subject":"[PATCH v2 2/2] p0005-status: time status on very large repo","fromName":"","fromEmail":"git@jeffhostetler.com","sentAt":"2017-04-05T21:09:20Z","receivedAt":"2017-04-05T21:09:44Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/p0005-status.sh | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++\n 1 file changed, 70 insertions(+)\n create mode 100644 t/perf/p0005-status.sh\n\ndiff --git a/t/perf/p0005-status.sh b/t/perf/p0005-status.sh\nnew file mode 100644\nindex 0000000..4a25ba0\n--- /dev/null\n+++ b/t/perf/p0005-status.sh\n@@ -0,0 +1,70 @@\n+#!/bin/sh\n+\n+test_description=\"Tests performance of read-tree\"\n+\n+. ./perf-lib.sh\n+\n+test_perf_default_repo\n+test_checkout_worktree\n+\n+## usage: dir depth width files\n+make_paths () {\n+\tfor f in $(seq $4)\n+\tdo\n+\t\techo $1/file$f\n+\tdone;\n+\tif test $2 -gt 0;\n+\tthen\n+\t\tfor w in $(seq $3)\n+\t\tdo\n+\t\t\tmake_paths $1/dir$w $(($2 - 1)) $3 $4\n+\t\tdone\n+\tfi\n+\treturn 0\n+}\n+\n+fill_index () {\n+\tmake_paths $1 $2 $3 $4 |\n+\tsed \"s/^/100644 $EMPTY_BLOB\t/\" |\n+\tgit update-index --index-info\n+\treturn 0\n+}\n+\n+br_work1=xxx_work1_xxx\n+\n+new_dir=xxx_dir_xxx\n+\n+## (5, 10, 9) will create 999,999 files.\n+## (4, 10, 9) will create 99,999 files.\n+depth=5\n+width=10\n+files=9\n+\n+export br_work1\n+\n+export new_dir\n+\n+export depth\n+export width\n+export files\n+\n+## Inflate the index with thousands of empty files and commit it.\n+test_expect_success 'inflate the index' '\n+\tgit reset --hard &&\n+\tgit branch $br_work1 &&\n+\tgit checkout $br_work1 &&\n+\tfill_index $new_dir $depth $width $files &&\n+\tgit commit -m $br_work1 &&\n+\tgit reset --hard\n+'\n+\n+## The number of files in the xxx_work1_xxx branch.\n+nr_work1=$(git ls-files | wc -l)\n+export nr_work1\n+\n+test_perf \"read-tree status work1 ($nr_work1)\" '\n+\tgit read-tree HEAD &&\n+\tgit status\n+'\n+\n+test_done\n-- \n2.9.3\n\n"},{"id":"316241","messageId":"20170405213548.GI8741@aiede.mtv.corp.google.com","threadId":"45611","inReplyTo":"20170405210920.56549-2-git@jeffhostetler.com","subject":"Re: [PATCH v2 1/2] string-list: use ALLOC_GROW macro when reallocing string_list","fromName":"Jonathan Nieder","fromEmail":"jrnieder@gmail.com","sentAt":"2017-04-05T21:35:48Z","receivedAt":"2017-04-05T21:35:56Z","isPatch":true,"sender":{"key":"jrnieder@gmail.com","avatar":"https://avatars.githubusercontent.com/u/281595?v=4"},"body":"git@jeffhostetler.com wrote:\n\n> During status on a very large repo and there are many changes,\n> a significant percentage of the total run time was spent\n\nnit: s/was/is/ (see comments on v1)\n\n> reallocing the wt_status.changes array.\n[...]\n> Signed-off-by: Jeff Hostetler \n> ---\n> string-list.c | 5 +----\n> 1 file changed, 1 insertion(+), 4 deletions(-)\n\nWith or without that wording tweak,\nReviewed-by: Jonathan Nieder \n"}]}