{"thread":{"id":"55087","subject":"[PATCH 01/11] p7519: use xargs -0 rather than -d in test","startedAt":"2021-02-01T22:03:15Z","lastAt":"2021-02-17T01:54:55Z","messageCount":30,"participants":["Jeff Hostetler via GitGitGadget","Kevin Willford via GitGitGadget","Junio C Hamano","Jeff Hostetler","Taylor Blau"],"isPatch":true,"patchVersion":1,"patchTotal":11},"messages":[{"id":"415817","messageId":"cf252e24b8c4da19ee9f886a1ab9c9c391d89d66.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 01/11] p7519: use xargs -0 rather than -d in test","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:10Z","receivedAt":"2021-02-01T22:03:15Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nThe Mac version of xargs does not support the \"-d\" option. Convert the test\nsetup to pipe the data set thru `lf_to_nul | xargs -0` instead.\n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/p7519-fsmonitor.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\nindex 9b43342806b..7bb37e9a6c1 100755\n--- a/t/perf/p7519-fsmonitor.sh\n+++ b/t/perf/p7519-fsmonitor.sh\n@@ -165,7 +165,7 @@ test_fsmonitor_suite() {\n \t'\n \n \ttest_perf_w_drop_caches \"status (dirty) ($DESC)\" '\n-\t\tgit ls-files | head -100000 | xargs -d \"\\n\" touch -h &&\n+\t\tgit ls-files | head -100000 | lf_to_nul | xargs -0 touch -h &&\n \t\tgit status\n \t'\n \n-- \ngitgitgadget\n\n"},{"id":"415818","messageId":"pull.860.git.1612216941.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":null,"subject":"[PATCH 00/11] FSMonitor Preliminary Commits","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:09Z","receivedAt":"2021-02-01T22:03:15Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"This patch series fixes runtime errors in t/perf/p7519 on Windows and MacOS.\nIt adds Trace2 logging to p7519 to make it easier to compare results when\nWatchman is enabled and disabled. And finally, it adds some Trace2 regions\nand data events around our usage of Watchman and the existing FSMonitor\nframework.\n\nThis series is independent of the \"Simple IPC\" series.\n\nA future series to add a builtin FSMonitor daemon will build upon both of\nthese series.\n\nJeff Hostetler (10):\n p7519: use xargs -0 rather than -d in test\n p7519: fix watchman watch-list test on Windows\n p7519: move watchman cleanup earlier in the test\n p7519: add trace logging during perf test\n preload-index: log the number of lstat calls to trace2\n read-cache: log the number of lstat calls to trace2\n read-cache: log the number of scanned files to trace2\n fsmonitor: log invocation of FSMonitor hook to trace2\n fsmonitor: log FSMN token when reading and writing the index\n fsmonitor: refactor initialization of fsmonitor_last_update token\n\nKevin Willford (1):\n fsmonitor: allow all entries for a folder to be invalidated\n\n fsmonitor.c | 107 ++++++++++++++++++++++++++++++++++----\n fsmonitor.h | 5 ++\n preload-index.c | 10 ++++\n read-cache.c | 24 +++++++--\n t/perf/.gitignore | 1 +\n t/perf/Makefile | 4 +-\n t/perf/p7519-fsmonitor.sh | 61 ++++++++++++++++++----\n 7 files changed, 186 insertions(+), 26 deletions(-)\n\n\nbase-commit: 71ca53e8125e36efbda17293c50027d31681a41f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-860%2Fjeffhostetler%2Ffsmonitor-prework-v1\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-860/jeffhostetler/fsmonitor-prework-v1\nPull-Request: https://github.com/gitgitgadget/git/pull/860\n-- \ngitgitgadget\n"},{"id":"415819","messageId":"a641f9e357c60a31176d85e14f2a7d7bc9006744.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 02/11] p7519: fix watchman watch-list test on Windows","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:11Z","receivedAt":"2021-02-01T22:03:15Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nOnly use the final portion of the test trash directory file name\nwhen verifying that Watchman was started.\n\nOn Windows and under the SDK, $GIT_WORKTREE is a cygwin-style\npath with forward slashes and a \"/c/\" drive name. However\n`watchman watch-list` reports a proper Windows-style pathname\nwith drive letters and backslashes. This causes the grep to\nfail. Since we don't really care about the full pathname (and\nwe really don't want to bother with normalizaing them), just see\nif the test-name portion of the path is found.\n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/p7519-fsmonitor.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\nindex 7bb37e9a6c1..e5a4b0582fb 100755\n--- a/t/perf/p7519-fsmonitor.sh\n+++ b/t/perf/p7519-fsmonitor.sh\n@@ -101,7 +101,7 @@ test_expect_success \"one time repo setup\" '\n \t# If Watchman exists, watch the work tree and attempt a query.\n \tif test_have_prereq WATCHMAN; then\n \t\twatchman watch \"$GIT_WORK_TREE\" &&\n-\t\twatchman watch-list | grep -q -F \"$GIT_WORK_TREE\"\n+\t\twatchman watch-list | grep -q -F \"p7519-fsmonitor\"\n \tfi\n '\n \n-- \ngitgitgadget\n\n"},{"id":"415820","messageId":"edb88ffe39e1f4e69432c0feb9b803d6cbe77f87.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 09/11] fsmonitor: log FSMN token when reading and writing the index","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:18Z","receivedAt":"2021-02-01T22:03:41Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nSigned-off-by: Jeff Hostetler \n---\n fsmonitor.c | 12 ++++++++++--\n 1 file changed, 10 insertions(+), 2 deletions(-)\n\ndiff --git a/fsmonitor.c b/fsmonitor.c\nindex 7a2be24cd43..3105dc370ab 100644\n--- a/fsmonitor.c\n+++ b/fsmonitor.c\n@@ -87,7 +87,11 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data,\n \t\tBUG(\"fsmonitor_dirty has more entries than the index (%\"PRIuMAX\" > %u)\",\n \t\t (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr);\n \n-\ttrace_printf_key(&trace_fsmonitor, \"read fsmonitor extension successful\");\n+\ttrace2_data_string(\"index\", NULL, \"extension/fsmn/read/token\",\n+\t\t\t istate->fsmonitor_last_update);\n+\ttrace_printf_key(&trace_fsmonitor,\n+\t\t\t \"read fsmonitor extension successful '%s'\",\n+\t\t\t istate->fsmonitor_last_update);\n \treturn 0;\n }\n \n@@ -133,7 +137,11 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate)\n \tput_be32(&ewah_size, sb->len - ewah_start);\n \tmemcpy(sb->buf + fixup, &ewah_size, sizeof(uint32_t));\n \n-\ttrace_printf_key(&trace_fsmonitor, \"write fsmonitor extension successful\");\n+\ttrace2_data_string(\"index\", NULL, \"extension/fsmn/write/token\",\n+\t\t\t istate->fsmonitor_last_update);\n+\ttrace_printf_key(&trace_fsmonitor,\n+\t\t\t \"write fsmonitor extension successful '%s'\",\n+\t\t\t istate->fsmonitor_last_update);\n }\n \n /*\n-- \ngitgitgadget\n\n"},{"id":"415821","messageId":"4686196bbc6a583eda16d2f2274e29df98284a22.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 11/11] fsmonitor: refactor initialization of fsmonitor_last_update token","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:20Z","receivedAt":"2021-02-01T22:03:41Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nIsolate and document initialization of `istate->fsmonitor_last_update`.\nThis field should contain a fsmonitor-specific opaque token, but we\nneed to initialize it before we can actually talk to a fsmonitor process,\nso we create a generic default value.\n\nSigned-off-by: Jeff Hostetler \n---\n fsmonitor.c | 35 ++++++++++++++++++++++++++++++++---\n 1 file changed, 32 insertions(+), 3 deletions(-)\n\ndiff --git a/fsmonitor.c b/fsmonitor.c\nindex 64deeda597e..e12214b3007 100644\n--- a/fsmonitor.c\n+++ b/fsmonitor.c\n@@ -343,16 +343,45 @@ void refresh_fsmonitor(struct index_state *istate)\n \tistate->fsmonitor_last_update = strbuf_detach(&last_update_token, NULL);\n }\n \n+/*\n+ * The caller wants to turn on FSMonitor. And when the caller writes\n+ * the index to disk, a FSMonitor extension should be included. This\n+ * requires that `istate->fsmonitor_last_update` not be NULL. But we\n+ * have not actually talked to a FSMonitor process yet, so we don't\n+ * have an initial value for this field.\n+ *\n+ * For a protocol V1 FSMonitor process, this field is a formatted\n+ * \"nanoseconds since epoch\" field. However, for a protocol V2\n+ * FSMonitor process, this field is an opaque token.\n+ *\n+ * Historically, `add_fsmonitor()` has initialized this field to the\n+ * current time for protocol V1 processes. There are lots of race\n+ * conditions here, but that code has shipped...\n+ *\n+ * The only true solution is to use a V2 FSMonitor and get a current\n+ * or default token value (that it understands), but we cannot do that\n+ * until we have actually talked to an instance of the FSMonitor process\n+ * (but the protocol requires that we send a token first...).\n+ *\n+ * For simplicity, just initialize like we have a V1 process and require\n+ * that V2 processes adapt.\n+ */\n+static void initialize_fsmonitor_last_update(struct index_state *istate)\n+{\n+\tstruct strbuf last_update = STRBUF_INIT;\n+\n+\tstrbuf_addf(&last_update, \"%\"PRIu64\"\", getnanotime());\n+\tistate->fsmonitor_last_update = strbuf_detach(&last_update, NULL);\n+}\n+\n void add_fsmonitor(struct index_state *istate)\n {\n \tunsigned int i;\n-\tstruct strbuf last_update = STRBUF_INIT;\n \n \tif (!istate->fsmonitor_last_update) {\n \t\ttrace_printf_key(&trace_fsmonitor, \"add fsmonitor\");\n \t\tistate->cache_changed |= FSMONITOR_CHANGED;\n-\t\tstrbuf_addf(&last_update, \"%\"PRIu64\"\", getnanotime());\n-\t\tistate->fsmonitor_last_update = strbuf_detach(&last_update, NULL);\n+\t\tinitialize_fsmonitor_last_update(istate);\n \n \t\t/* reset the fsmonitor state */\n \t\tfor (i = 0; i < istate->cache_nr; i++)\n-- \ngitgitgadget\n"},{"id":"415822","messageId":"2af6858716f161a12acda7f07e74b0dfed03101b.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 03/11] p7519: move watchman cleanup earlier in the test","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:12Z","receivedAt":"2021-02-01T22:03:45Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nShutdown Watchman after the Watchman-based tests and before the block of\n\"no fsmonitor\" tests.\n\nThis helps ensure that Watchman cannot affect the test results for the\nother.\n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/p7519-fsmonitor.sh | 25 +++++++++++++++++--------\n 1 file changed, 17 insertions(+), 8 deletions(-)\n\ndiff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\nindex e5a4b0582fb..45bbba3c92f 100755\n--- a/t/perf/p7519-fsmonitor.sh\n+++ b/t/perf/p7519-fsmonitor.sh\n@@ -198,6 +198,11 @@ test_fsmonitor_suite() {\n \t'\n }\n \n+#\n+# Run a full set of perf tests using each Hook-based fsmonitor provider,\n+# such as Watchman.\n+#\n+\n if test -n \"$GIT_PERF_7519_FSMONITOR\"; then\n \tfor INTEGRATION_PATH in $GIT_PERF_7519_FSMONITOR; do\n \t\ttest_expect_success \"setup for fsmonitor $INTEGRATION_PATH\" 'setup_for_fsmonitor'\n@@ -208,14 +213,6 @@ else\n \ttest_fsmonitor_suite\n fi\n \n-test_expect_success \"setup without fsmonitor\" '\n-\tunset INTEGRATION_SCRIPT &&\n-\tgit config --unset core.fsmonitor &&\n-\tgit update-index --no-fsmonitor\n-'\n-\n-test_fsmonitor_suite\n-\n if test_have_prereq WATCHMAN\n then\n \twatchman watch-del \"$GIT_WORK_TREE\" >/dev/null 2>&1 &&\n@@ -225,4 +222,16 @@ then\n \twatchman shutdown-server >/dev/null 2>&1\n fi\n \n+#\n+# Run a full set of perf tests with the fsmonitor feature disabled.\n+#\n+\n+test_expect_success \"setup without fsmonitor\" '\n+\tunset INTEGRATION_SCRIPT &&\n+\tgit config --unset core.fsmonitor &&\n+\tgit update-index --no-fsmonitor\n+'\n+\n+test_fsmonitor_suite\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"415823","messageId":"8de9985a706675ff41a3549ce00dd42a9dc92c88.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 04/11] p7519: add trace logging during perf test","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:13Z","receivedAt":"2021-02-01T22:03:47Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nAdd optional trace logging to allow us to better compare performance of\nvarious fsmonitor providers and compare results with non-fsmonitor runs.\n\nCurrently, this includes Trace2 logging, but may be extended to include\nother trace targets, such as GIT_TRACE_FSMONITOR if desired.\n\nUsing this logging helped me explain an odd behavior on MacOS where the\nkernel was dropping events and causing the hook to Watchman to timeout.\n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/.gitignore | 1 +\n t/perf/Makefile | 4 ++--\n t/perf/p7519-fsmonitor.sh | 32 ++++++++++++++++++++++++++++++++\n 3 files changed, 35 insertions(+), 2 deletions(-)\n\ndiff --git a/t/perf/.gitignore b/t/perf/.gitignore\nindex 982eb8e3a94..72f5d0d3148 100644\n--- a/t/perf/.gitignore\n+++ b/t/perf/.gitignore\n@@ -1,3 +1,4 @@\n /build/\n /test-results/\n+/test-trace/\n /trash directory*/\ndiff --git a/t/perf/Makefile b/t/perf/Makefile\nindex fcb0e8865e4..2465770a782 100644\n--- a/t/perf/Makefile\n+++ b/t/perf/Makefile\n@@ -7,10 +7,10 @@ perf: pre-clean\n \t./run\n \n pre-clean:\n-\trm -rf test-results\n+\trm -rf test-results test-trace\n \n clean:\n-\trm -rf build \"trash directory\".* test-results\n+\trm -rf build \"trash directory\".* test-results test-trace\n \n test-lint:\n \t$(MAKE) -C .. test-lint\ndiff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\nindex 45bbba3c92f..e6724d3604b 100755\n--- a/t/perf/p7519-fsmonitor.sh\n+++ b/t/perf/p7519-fsmonitor.sh\n@@ -32,6 +32,8 @@ test_description=\"Test core.fsmonitor\"\n #\n # GIT_PERF_7519_DROP_CACHE: if set, the OS caches are dropped between tests\n #\n+# GIT_PERF_7519_TRACE: if set, enable trace logging during the test.\n+# Trace logs will be grouped by fsmonitor provider.\n \n test_perf_large_repo\n test_checkout_worktree\n@@ -70,6 +72,32 @@ then\n \tfi\n fi\n \n+trace_start() {\n+\tif test -n \"$GIT_PERF_7519_TRACE\"\n+\tthen\n+\t\tname=\"$1\"\n+\t\tTEST_TRACE_DIR=\"$TEST_OUTPUT_DIRECTORY/test-trace/p7519/\"\n+\t\techo \"Writing trace logging to $TEST_TRACE_DIR\"\n+\n+\t\tmkdir -p \"$TEST_TRACE_DIR\"\n+\n+\t\t# Start Trace2 logging and any other GIT_TRACE_* logs that you\n+\t\t# want for this named test case.\n+\n+\t\tGIT_TRACE2_PERF=\"$TEST_TRACE_DIR/$name.trace2perf\"\n+\t\texport GIT_TRACE2_PERF\n+\n+\t\t>\"$GIT_TRACE2_PERF\"\n+\tfi\n+}\n+\n+trace_stop() {\n+\tif test -n \"$GIT_PERF_7519_TRACE\"\n+\tthen\n+\t\tunset GIT_TRACE2_PERF\n+\tfi\n+}\n+\n test_expect_success \"one time repo setup\" '\n \t# set untrackedCache depending on the environment\n \tif test -n \"$GIT_PERF_7519_UNTRACKED_CACHE\"\n@@ -203,6 +231,7 @@ test_fsmonitor_suite() {\n # such as Watchman.\n #\n \n+trace_start fsmonitor-watchman\n if test -n \"$GIT_PERF_7519_FSMONITOR\"; then\n \tfor INTEGRATION_PATH in $GIT_PERF_7519_FSMONITOR; do\n \t\ttest_expect_success \"setup for fsmonitor $INTEGRATION_PATH\" 'setup_for_fsmonitor'\n@@ -221,11 +250,13 @@ then\n \t# preventing the removal of the trash directory\n \twatchman shutdown-server >/dev/null 2>&1\n fi\n+trace_stop\n \n #\n # Run a full set of perf tests with the fsmonitor feature disabled.\n #\n \n+trace_start fsmonitor-disabled\n test_expect_success \"setup without fsmonitor\" '\n \tunset INTEGRATION_SCRIPT &&\n \tgit config --unset core.fsmonitor &&\n@@ -233,5 +264,6 @@ test_expect_success \"setup without fsmonitor\" '\n '\n \n test_fsmonitor_suite\n+trace_stop\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"415824","messageId":"cdd49f1fdb18b85430c39f65d2a85890abf927e7.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 05/11] preload-index: log the number of lstat calls to trace2","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:14Z","receivedAt":"2021-02-01T22:03:56Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nReport the total number of calls made to lstat() inside preload_index().\n\nFSMonitor improves the performance of commands like `git status` by\navoiding scanning the disk for changed files. This can be seen in\n`preload_index()`. Let's measure this.\n\nSigned-off-by: Jeff Hostetler \n---\n preload-index.c | 10 ++++++++++\n 1 file changed, 10 insertions(+)\n\ndiff --git a/preload-index.c b/preload-index.c\nindex ed6eaa47388..e5529a58636 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -31,6 +31,7 @@ struct thread_data {\n \tstruct pathspec pathspec;\n \tstruct progress_data *progress;\n \tint offset, nr;\n+\tint t2_nr_lstat;\n };\n \n static void *preload_thread(void *_data)\n@@ -73,6 +74,7 @@ static void *preload_thread(void *_data)\n \t\t\tcontinue;\n \t\tif (threaded_has_symlink_leading_path(&cache, ce->name, ce_namelen(ce)))\n \t\t\tcontinue;\n+\t\tp->t2_nr_lstat++;\n \t\tif (lstat(ce->name, &st))\n \t\t\tcontinue;\n \t\tif (ie_match_stat(index, ce, &st, CE_MATCH_RACY_IS_DIRTY|CE_MATCH_IGNORE_FSMONITOR))\n@@ -98,6 +100,7 @@ void preload_index(struct index_state *index,\n \tint threads, i, work, offset;\n \tstruct thread_data data[MAX_PARALLEL];\n \tstruct progress_data pd;\n+\tint t2_sum_lstat = 0;\n \n \tif (!HAVE_THREADS || !core_preload_index)\n \t\treturn;\n@@ -107,6 +110,9 @@ void preload_index(struct index_state *index,\n \t\tthreads = 2;\n \tif (threads < 2)\n \t\treturn;\n+\n+\ttrace2_region_enter(\"index\", \"preload\", NULL);\n+\n \ttrace_performance_enter();\n \tif (threads > MAX_PARALLEL)\n \t\tthreads = MAX_PARALLEL;\n@@ -141,10 +147,14 @@ void preload_index(struct index_state *index,\n \t\tstruct thread_data *p = data+i;\n \t\tif (pthread_join(p->pthread, NULL))\n \t\t\tdie(\"unable to join threaded lstat\");\n+\t\tt2_sum_lstat += p->t2_nr_lstat;\n \t}\n \tstop_progress(&pd.progress);\n \n \ttrace_performance_leave(\"preload index\");\n+\n+\ttrace2_data_intmax(\"index\", NULL, \"preload/sum_lstat\", t2_sum_lstat);\n+\ttrace2_region_leave(\"index\", \"preload\", NULL);\n }\n \n int repo_read_index_preload(struct repository *repo,\n-- \ngitgitgadget\n\n"},{"id":"415825","messageId":"c84531f624428c19408fb1f05343f27ee88e2654.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 07/11] read-cache: log the number of scanned files to trace2","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:16Z","receivedAt":"2021-02-01T22:04:00Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nReport the number of files in the working directory that were read and\ntheir hashes verified in `refresh_index()`.\n\nFSMonitor improves the performance of commands like `git status` by\navoiding scanning the disk for changed files. Let's measure this.\n\nSigned-off-by: Jeff Hostetler \n---\n read-cache.c | 13 ++++++++++---\n 1 file changed, 10 insertions(+), 3 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 893cc41e1d9..c9dd7f4015e 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -1365,7 +1365,8 @@ static struct cache_entry *refresh_cache_ent(struct index_state *istate,\n \t\t\t\t\t struct cache_entry *ce,\n \t\t\t\t\t unsigned int options, int *err,\n \t\t\t\t\t int *changed_ret,\n-\t\t\t\t\t int *t2_did_lstat)\n+\t\t\t\t\t int *t2_did_lstat,\n+\t\t\t\t\t int *t2_did_scan)\n {\n \tstruct stat st;\n \tstruct cache_entry *updated;\n@@ -1445,6 +1446,8 @@ static struct cache_entry *refresh_cache_ent(struct index_state *istate,\n \t\t}\n \t}\n \n+\tif (t2_did_scan)\n+\t\t*t2_did_scan = 1;\n \tif (ie_modified(istate, ce, &st, options)) {\n \t\tif (err)\n \t\t\t*err = EINVAL;\n@@ -1523,6 +1526,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \tconst char *unmerged_fmt;\n \tstruct progress *progress = NULL;\n \tint t2_sum_lstat = 0;\n+\tint t2_sum_scan = 0;\n \n \tif (flags & REFRESH_PROGRESS && isatty(2))\n \t\tprogress = start_delayed_progress(_(\"Refresh index\"),\n@@ -1547,6 +1551,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \t\tint changed = 0;\n \t\tint filtered = 0;\n \t\tint t2_did_lstat = 0;\n+\t\tint t2_did_scan = 0;\n \n \t\tce = istate->cache[i];\n \t\tif (ignore_submodules && S_ISGITLINK(ce->ce_mode))\n@@ -1574,8 +1579,9 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \n \t\tnew_entry = refresh_cache_ent(istate, ce, options,\n \t\t\t\t\t &cache_errno, &changed,\n-\t\t\t\t\t &t2_did_lstat);\n+\t\t\t\t\t &t2_did_lstat, &t2_did_scan);\n \t\tt2_sum_lstat += t2_did_lstat;\n+\t\tt2_sum_scan += t2_did_scan;\n \t\tif (new_entry == ce)\n \t\t\tcontinue;\n \t\tif (progress)\n@@ -1612,6 +1618,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \t\treplace_index_entry(istate, i, new_entry);\n \t}\n \ttrace2_data_intmax(\"index\", NULL, \"refresh/sum_lstat\", t2_sum_lstat);\n+\ttrace2_data_intmax(\"index\", NULL, \"refresh/sum_scan\", t2_sum_scan);\n \ttrace2_region_leave(\"index\", \"refresh\", NULL);\n \tif (progress) {\n \t\tdisplay_progress(progress, istate->cache_nr);\n@@ -1625,7 +1632,7 @@ struct cache_entry *refresh_cache_entry(struct index_state *istate,\n \t\t\t\t\tstruct cache_entry *ce,\n \t\t\t\t\tunsigned int options)\n {\n-\treturn refresh_cache_ent(istate, ce, options, NULL, NULL, NULL);\n+\treturn refresh_cache_ent(istate, ce, options, NULL, NULL, NULL, NULL);\n }\n \n \n-- \ngitgitgadget\n\n"},{"id":"415826","messageId":"384d2eff8637b7dd4cfbb116dd4b3ce30de5e381.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 10/11] fsmonitor: allow all entries for a folder to be invalidated","fromName":"Kevin Willford via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:19Z","receivedAt":"2021-02-01T22:04:02Z","isPatch":true,"sender":{"key":"kewillf@microsoft.com","avatar":null},"body":"From: Kevin Willford \n\nAllow fsmonitor to report directory changes by reporting paths with a\ntrailing slash.\n\nSigned-off-by: Jeff Hostetler \nSigned-off-by: Kevin Willford \nSigned-off-by: Johannes Schindelin \n---\n fsmonitor.c | 31 ++++++++++++++++++++++++++-----\n 1 file changed, 26 insertions(+), 5 deletions(-)\n\ndiff --git a/fsmonitor.c b/fsmonitor.c\nindex 3105dc370ab..64deeda597e 100644\n--- a/fsmonitor.c\n+++ b/fsmonitor.c\n@@ -190,13 +190,34 @@ int fsmonitor_is_trivial_response(const struct strbuf *query_result)\n \treturn is_trivial;\n }\n \n-static void fsmonitor_refresh_callback(struct index_state *istate, const char *name)\n+static void fsmonitor_refresh_callback(struct index_state *istate, char *name)\n {\n-\tint pos = index_name_pos(istate, name, strlen(name));\n+\tint i, len = strlen(name);\n+\tif (name[len - 1] == '/') {\n+\n+\t\t/*\n+\t\t * TODO We should binary search to find the first path with\n+\t\t * TODO this directory prefix. Then linearly update entries\n+\t\t * TODO while the prefix matches. Taking care to search without\n+\t\t * TODO the trailing slash -- because '/' sorts after a few\n+\t\t * TODO interesting special chars, like '.' and ' '.\n+\t\t */\n+\n+\t\t/* Mark all entries for the folder invalid */\n+\t\tfor (i = 0; i < istate->cache_nr; i++) {\n+\t\t\tif (istate->cache[i]->ce_flags & CE_FSMONITOR_VALID &&\n+\t\t\t starts_with(istate->cache[i]->name, name))\n+\t\t\t\tistate->cache[i]->ce_flags &= ~CE_FSMONITOR_VALID;\n+\t\t}\n+\t\t/* Need to remove the / from the path for the untracked cache */\n+\t\tname[len - 1] = '\\0';\n+\t} else {\n+\t\tint pos = index_name_pos(istate, name, strlen(name));\n \n-\tif (pos >= 0) {\n-\t\tstruct cache_entry *ce = istate->cache[pos];\n-\t\tce->ce_flags &= ~CE_FSMONITOR_VALID;\n+\t\tif (pos >= 0) {\n+\t\t\tstruct cache_entry *ce = istate->cache[pos];\n+\t\t\tce->ce_flags &= ~CE_FSMONITOR_VALID;\n+\t\t}\n \t}\n \n \t/*\n-- \ngitgitgadget\n\n"},{"id":"415827","messageId":"ef64b60c7a0c85df08198a8f104029742c555bad.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 08/11] fsmonitor: log invocation of FSMonitor hook to trace2","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:17Z","receivedAt":"2021-02-01T22:04:26Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nLet's measure the time taken to request and receive FSMonitor data\nvia the hook API and the size of the response.\n\nSigned-off-by: Jeff Hostetler \n---\n fsmonitor.c | 29 ++++++++++++++++++++++++++++-\n fsmonitor.h | 5 +++++\n 2 files changed, 33 insertions(+), 1 deletion(-)\n\ndiff --git a/fsmonitor.c b/fsmonitor.c\nindex ca031c3abb8..7a2be24cd43 100644\n--- a/fsmonitor.c\n+++ b/fsmonitor.c\n@@ -142,6 +142,7 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate)\n static int query_fsmonitor(int version, const char *last_update, struct strbuf *query_result)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tint result;\n \n \tif (!core_fsmonitor)\n \t\treturn -1;\n@@ -152,7 +153,33 @@ static int query_fsmonitor(int version, const char *last_update, struct strbuf *\n \tcp.use_shell = 1;\n \tcp.dir = get_git_work_tree();\n \n-\treturn capture_command(&cp, query_result, 1024);\n+\ttrace2_region_enter(\"fsm_hook\", \"query\", NULL);\n+\n+\tresult = capture_command(&cp, query_result, 1024);\n+\n+\tif (result)\n+\t\ttrace2_data_intmax(\"fsm_hook\", NULL, \"query/failed\", result);\n+\telse {\n+\t\ttrace2_data_intmax(\"fsm_hook\", NULL, \"query/response-length\",\n+\t\t\t\t query_result->len);\n+\n+\t\tif (fsmonitor_is_trivial_response(query_result))\n+\t\t\ttrace2_data_intmax(\"fsm_hook\", NULL,\n+\t\t\t\t\t \"query/trivial-response\", 1);\n+\t}\n+\n+\ttrace2_region_leave(\"fsm_hook\", \"query\", NULL);\n+\n+\treturn result;\n+}\n+\n+int fsmonitor_is_trivial_response(const struct strbuf *query_result)\n+{\n+\tstatic char trivial_response[3] = { '\\0', '/', '\\0' };\n+\tint is_trivial = !memcmp(trivial_response,\n+\t\t\t\t &query_result->buf[query_result->len - 3], 3);\n+\n+\treturn is_trivial;\n }\n \n static void fsmonitor_refresh_callback(struct index_state *istate, const char *name)\ndiff --git a/fsmonitor.h b/fsmonitor.h\nindex 739318ab6d1..7f1794b90b0 100644\n--- a/fsmonitor.h\n+++ b/fsmonitor.h\n@@ -44,6 +44,11 @@ void tweak_fsmonitor(struct index_state *istate);\n */\n void refresh_fsmonitor(struct index_state *istate);\n \n+/*\n+ * Does the received result contain the \"trivial\" response?\n+ */\n+int fsmonitor_is_trivial_response(const struct strbuf *query_result);\n+\n /*\n * Set the given cache entries CE_FSMONITOR_VALID bit. This should be\n * called any time the cache entry has been updated to reflect the\n-- \ngitgitgadget\n\n"},{"id":"415828","messageId":"65488f7a1bfa8313a7b7d59e208e54cb7f7c18ae.1612216941.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH 06/11] read-cache: log the number of lstat calls to trace2","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-01T22:02:15Z","receivedAt":"2021-02-01T22:04:31Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nReport the total number of calls made to lstat() inside of refresh_index().\n\nFSMonitor improves the performance of commands like `git status` by\navoiding scanning the disk for changed files. This can be seen in\n`refresh_index()`. Let's measure this.\n\nSigned-off-by: Jeff Hostetler \n---\n read-cache.c | 17 ++++++++++++++---\n 1 file changed, 14 insertions(+), 3 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex ecf6f689940..893cc41e1d9 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -1364,7 +1364,8 @@ int add_index_entry(struct index_state *istate, struct cache_entry *ce, int opti\n static struct cache_entry *refresh_cache_ent(struct index_state *istate,\n \t\t\t\t\t struct cache_entry *ce,\n \t\t\t\t\t unsigned int options, int *err,\n-\t\t\t\t\t int *changed_ret)\n+\t\t\t\t\t int *changed_ret,\n+\t\t\t\t\t int *t2_did_lstat)\n {\n \tstruct stat st;\n \tstruct cache_entry *updated;\n@@ -1406,6 +1407,8 @@ static struct cache_entry *refresh_cache_ent(struct index_state *istate,\n \t\treturn NULL;\n \t}\n \n+\tif (t2_did_lstat)\n+\t\t*t2_did_lstat = 1;\n \tif (lstat(ce->name, &st) < 0) {\n \t\tif (ignore_missing && errno == ENOENT)\n \t\t\treturn ce;\n@@ -1519,6 +1522,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \tconst char *added_fmt;\n \tconst char *unmerged_fmt;\n \tstruct progress *progress = NULL;\n+\tint t2_sum_lstat = 0;\n \n \tif (flags & REFRESH_PROGRESS && isatty(2))\n \t\tprogress = start_delayed_progress(_(\"Refresh index\"),\n@@ -1536,11 +1540,13 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \t * we only have to do the special cases that are left.\n \t */\n \tpreload_index(istate, pathspec, 0);\n+\ttrace2_region_enter(\"index\", \"refresh\", NULL);\n \tfor (i = 0; i < istate->cache_nr; i++) {\n \t\tstruct cache_entry *ce, *new_entry;\n \t\tint cache_errno = 0;\n \t\tint changed = 0;\n \t\tint filtered = 0;\n+\t\tint t2_did_lstat = 0;\n \n \t\tce = istate->cache[i];\n \t\tif (ignore_submodules && S_ISGITLINK(ce->ce_mode))\n@@ -1566,7 +1572,10 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \t\tif (filtered)\n \t\t\tcontinue;\n \n-\t\tnew_entry = refresh_cache_ent(istate, ce, options, &cache_errno, &changed);\n+\t\tnew_entry = refresh_cache_ent(istate, ce, options,\n+\t\t\t\t\t &cache_errno, &changed,\n+\t\t\t\t\t &t2_did_lstat);\n+\t\tt2_sum_lstat += t2_did_lstat;\n \t\tif (new_entry == ce)\n \t\t\tcontinue;\n \t\tif (progress)\n@@ -1602,6 +1611,8 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \n \t\treplace_index_entry(istate, i, new_entry);\n \t}\n+\ttrace2_data_intmax(\"index\", NULL, \"refresh/sum_lstat\", t2_sum_lstat);\n+\ttrace2_region_leave(\"index\", \"refresh\", NULL);\n \tif (progress) {\n \t\tdisplay_progress(progress, istate->cache_nr);\n \t\tstop_progress(&progress);\n@@ -1614,7 +1625,7 @@ struct cache_entry *refresh_cache_entry(struct index_state *istate,\n \t\t\t\t\tstruct cache_entry *ce,\n \t\t\t\t\tunsigned int options)\n {\n-\treturn refresh_cache_ent(istate, ce, options, NULL, NULL);\n+\treturn refresh_cache_ent(istate, ce, options, NULL, NULL, NULL);\n }\n \n \n-- \ngitgitgadget\n\n"},{"id":"415844","messageId":"xmqqsg6fid57.fsf@gitster.c.googlers.com","threadId":"55087","inReplyTo":"cf252e24b8c4da19ee9f886a1ab9c9c391d89d66.1612216941.git.gitgitgadget@gmail.com","subject":"Re: [PATCH 01/11] p7519: use xargs -0 rather than -d in test","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-01T23:25:40Z","receivedAt":"2021-02-01T23:26:42Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"\"Jeff Hostetler via GitGitGadget\" writes:\n\n> From: Jeff Hostetler \n>\n> The Mac version of xargs does not support the \"-d\" option. Convert the test\n> setup to pipe the data set thru `lf_to_nul | xargs -0` instead.\n\n\"xargs -0\" is not all that portable, either, and neither is \"touch -h\".\n\nBut since the t/perf stuff already depends on having GNU toolchain\nanyway, I can be persuaded to believe that it is OK.\n\nDo we know that this part runs much later than the staged files are\nlast touched, so that these uses of \"touch\" actually are effective\nto make the paths stat-dirty? Otherwise, we may be just \"touch\"ing\nthem with the timestamp they already have after all.\n\nThanks.\n\n> Signed-off-by: Jeff Hostetler \n> ---\n> t/perf/p7519-fsmonitor.sh | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n>\n> diff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\n> index 9b43342806b..7bb37e9a6c1 100755\n> --- a/t/perf/p7519-fsmonitor.sh\n> +++ b/t/perf/p7519-fsmonitor.sh\n> @@ -165,7 +165,7 @@ test_fsmonitor_suite() {\n> \t'\n> \n> \ttest_perf_w_drop_caches \"status (dirty) ($DESC)\" '\n> -\t\tgit ls-files | head -100000 | xargs -d \"\\n\" touch -h &&\n> +\t\tgit ls-files | head -100000 | lf_to_nul | xargs -0 touch -h &&\n> \t\tgit status\n> \t'\n"},{"id":"415945","messageId":"b2c548b7-f641-2398-c200-d774af9cf628@jeffhostetler.com","threadId":"55087","inReplyTo":"xmqqsg6fid57.fsf@gitster.c.googlers.com","subject":"Re: [PATCH 01/11] p7519: use xargs -0 rather than -d in test","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-02-02T18:16:36Z","receivedAt":"2021-02-02T18:22:05Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 2/1/21 6:25 PM, Junio C Hamano wrote:\n> \"Jeff Hostetler via GitGitGadget\" writes:\n> \n>> From: Jeff Hostetler \n>>\n>> The Mac version of xargs does not support the \"-d\" option. Convert the test\n>> setup to pipe the data set thru `lf_to_nul | xargs -0` instead.\n> \n> \"xargs -0\" is not all that portable, either, and neither is \"touch -h\".\n> \n> But since the t/perf stuff already depends on having GNU toolchain\n> anyway, I can be persuaded to believe that it is OK.\n> \n> Do we know that this part runs much later than the staged files are\n> last touched, so that these uses of \"touch\" actually are effective\n> to make the paths stat-dirty? Otherwise, we may be just \"touch\"ing\n> them with the timestamp they already have after all.\n\nI'm not sure now that you mention it. I suppose on modern filesystems\nthat have mtimes with nanosecond fields we could (are) assuming that\n\"touch\" is actually doing something. On older filesystems (such as\nFAT32), you're right it is probably not doing anything at the speed\nthat the test runs.\n\nTBH I'm not sure that the test needs the \"-h\". Symlinks are not that\ncommon and it shouldn't affect the timings that much if there are a few.\n\nI'm not sure what to do about \"-0\". Not even \"--null\" is portable.\n\nLet me do a little digging here.\n\n> \n> Thanks.\n> \n>> Signed-off-by: Jeff Hostetler \n>> ---\n>> t/perf/p7519-fsmonitor.sh | 2 +-\n>> 1 file changed, 1 insertion(+), 1 deletion(-)\n>>\n>> diff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\n>> index 9b43342806b..7bb37e9a6c1 100755\n>> --- a/t/perf/p7519-fsmonitor.sh\n>> +++ b/t/perf/p7519-fsmonitor.sh\n>> @@ -165,7 +165,7 @@ test_fsmonitor_suite() {\n>> \t'\n>> \n>> \ttest_perf_w_drop_caches \"status (dirty) ($DESC)\" '\n>> -\t\tgit ls-files | head -100000 | xargs -d \"\\n\" touch -h &&\n>> +\t\tgit ls-files | head -100000 | lf_to_nul | xargs -0 touch -h &&\n>> \t\tgit status\n>> \t'\n"},{"id":"415960","messageId":"xmqqpn1if9jt.fsf@gitster.c.googlers.com","threadId":"55087","inReplyTo":"b2c548b7-f641-2398-c200-d774af9cf628@jeffhostetler.com","subject":"Re: [PATCH 01/11] p7519: use xargs -0 rather than -d in test","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-02T21:23:50Z","receivedAt":"2021-02-02T21:25:10Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff Hostetler writes:\n\n> I'm not sure now that you mention it. I suppose on modern filesystems\n> that have mtimes with nanosecond fields we could (are) assuming that\n> \"touch\" is actually doing something. On older filesystems (such as\n> FAT32), you're right it is probably not doing anything at the speed\n> that the test runs.\n\nThat one is probably the most relevant nit among the ones I raised.\nI do not actually mind if we used test-chmtime to force our own\ntimestamp (e.g. \"5 seconds before the filesystem time\"), and added\nthe helper the \"--stdin\" option to read paths to work around the\n\"xargs\" issue.\n\n> TBH I'm not sure that the test needs the \"-h\". Symlinks are not that\n> common and it shouldn't affect the timings that much if there are a few.\n\nI agree.\n\n> I'm not sure what to do about \"-0\". Not even \"--null\" is portable.\n\nCorrect. I do not think it is worth \"digging\", though. I do not\nmind \"ls-files -z | test-tool chmtime -600 --stdin -z\" to lose\nxargs, but we already depend on GNU time to run t/perf, and it is\nnot too far a stretch to require GNU xargs that knows \"-0\" or \"-d\".\n\nThanks.\n\n"},{"id":"416045","messageId":"9ceba5e69420f12eeb834f387372520b061e28a7.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 03/11] p7519: move watchman cleanup earlier in the test","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:42Z","receivedAt":"2021-02-03T15:38:14Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nShutdown Watchman after the Watchman-based tests and before the block of\n\"no fsmonitor\" tests.\n\nThis helps ensure that Watchman cannot affect the test results for the\nother.\n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/p7519-fsmonitor.sh | 25 +++++++++++++++++--------\n 1 file changed, 17 insertions(+), 8 deletions(-)\n\ndiff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\nindex 21d525541d5..78e7d2c03f5 100755\n--- a/t/perf/p7519-fsmonitor.sh\n+++ b/t/perf/p7519-fsmonitor.sh\n@@ -208,6 +208,11 @@ test_fsmonitor_suite() {\n \t'\n }\n \n+#\n+# Run a full set of perf tests using each Hook-based fsmonitor provider,\n+# such as Watchman.\n+#\n+\n if test -n \"$GIT_PERF_7519_FSMONITOR\"; then\n \tfor INTEGRATION_PATH in $GIT_PERF_7519_FSMONITOR; do\n \t\ttest_expect_success \"setup for fsmonitor $INTEGRATION_PATH\" 'setup_for_fsmonitor'\n@@ -218,14 +223,6 @@ else\n \ttest_fsmonitor_suite\n fi\n \n-test_expect_success \"setup without fsmonitor\" '\n-\tunset INTEGRATION_SCRIPT &&\n-\tgit config --unset core.fsmonitor &&\n-\tgit update-index --no-fsmonitor\n-'\n-\n-test_fsmonitor_suite\n-\n if test_have_prereq WATCHMAN\n then\n \twatchman watch-del \"$GIT_WORK_TREE\" >/dev/null 2>&1 &&\n@@ -235,4 +232,16 @@ then\n \twatchman shutdown-server >/dev/null 2>&1\n fi\n \n+#\n+# Run a full set of perf tests with the fsmonitor feature disabled.\n+#\n+\n+test_expect_success \"setup without fsmonitor\" '\n+\tunset INTEGRATION_SCRIPT &&\n+\tgit config --unset core.fsmonitor &&\n+\tgit update-index --no-fsmonitor\n+'\n+\n+test_fsmonitor_suite\n+\n test_done\n-- \ngitgitgadget\n\n"},{"id":"416046","messageId":"f6ea0a51f50e95d943a33156c3013af97ddda470.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 04/11] p7519: add trace logging during perf test","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:43Z","receivedAt":"2021-02-03T15:38:51Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nAdd optional trace logging to allow us to better compare performance of\nvarious fsmonitor providers and compare results with non-fsmonitor runs.\n\nCurrently, this includes Trace2 logging, but may be extended to include\nother trace targets, such as GIT_TRACE_FSMONITOR if desired.\n\nUsing this logging helped me explain an odd behavior on MacOS where the\nkernel was dropping events and causing the hook to Watchman to timeout.\n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/.gitignore | 1 +\n t/perf/Makefile | 4 ++--\n t/perf/p7519-fsmonitor.sh | 32 ++++++++++++++++++++++++++++++++\n 3 files changed, 35 insertions(+), 2 deletions(-)\n\ndiff --git a/t/perf/.gitignore b/t/perf/.gitignore\nindex 982eb8e3a94..72f5d0d3148 100644\n--- a/t/perf/.gitignore\n+++ b/t/perf/.gitignore\n@@ -1,3 +1,4 @@\n /build/\n /test-results/\n+/test-trace/\n /trash directory*/\ndiff --git a/t/perf/Makefile b/t/perf/Makefile\nindex fcb0e8865e4..2465770a782 100644\n--- a/t/perf/Makefile\n+++ b/t/perf/Makefile\n@@ -7,10 +7,10 @@ perf: pre-clean\n \t./run\n \n pre-clean:\n-\trm -rf test-results\n+\trm -rf test-results test-trace\n \n clean:\n-\trm -rf build \"trash directory\".* test-results\n+\trm -rf build \"trash directory\".* test-results test-trace\n \n test-lint:\n \t$(MAKE) -C .. test-lint\ndiff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\nindex 78e7d2c03f5..aa0ea0e2ec8 100755\n--- a/t/perf/p7519-fsmonitor.sh\n+++ b/t/perf/p7519-fsmonitor.sh\n@@ -32,6 +32,8 @@ test_description=\"Test core.fsmonitor\"\n #\n # GIT_PERF_7519_DROP_CACHE: if set, the OS caches are dropped between tests\n #\n+# GIT_PERF_7519_TRACE: if set, enable trace logging during the test.\n+# Trace logs will be grouped by fsmonitor provider.\n \n test_perf_large_repo\n test_checkout_worktree\n@@ -70,6 +72,32 @@ then\n \tfi\n fi\n \n+trace_start() {\n+\tif test -n \"$GIT_PERF_7519_TRACE\"\n+\tthen\n+\t\tname=\"$1\"\n+\t\tTEST_TRACE_DIR=\"$TEST_OUTPUT_DIRECTORY/test-trace/p7519/\"\n+\t\techo \"Writing trace logging to $TEST_TRACE_DIR\"\n+\n+\t\tmkdir -p \"$TEST_TRACE_DIR\"\n+\n+\t\t# Start Trace2 logging and any other GIT_TRACE_* logs that you\n+\t\t# want for this named test case.\n+\n+\t\tGIT_TRACE2_PERF=\"$TEST_TRACE_DIR/$name.trace2perf\"\n+\t\texport GIT_TRACE2_PERF\n+\n+\t\t>\"$GIT_TRACE2_PERF\"\n+\tfi\n+}\n+\n+trace_stop() {\n+\tif test -n \"$GIT_PERF_7519_TRACE\"\n+\tthen\n+\t\tunset GIT_TRACE2_PERF\n+\tfi\n+}\n+\n test_expect_success \"one time repo setup\" '\n \t# set untrackedCache depending on the environment\n \tif test -n \"$GIT_PERF_7519_UNTRACKED_CACHE\"\n@@ -213,6 +241,7 @@ test_fsmonitor_suite() {\n # such as Watchman.\n #\n \n+trace_start fsmonitor-watchman\n if test -n \"$GIT_PERF_7519_FSMONITOR\"; then\n \tfor INTEGRATION_PATH in $GIT_PERF_7519_FSMONITOR; do\n \t\ttest_expect_success \"setup for fsmonitor $INTEGRATION_PATH\" 'setup_for_fsmonitor'\n@@ -231,11 +260,13 @@ then\n \t# preventing the removal of the trash directory\n \twatchman shutdown-server >/dev/null 2>&1\n fi\n+trace_stop\n \n #\n # Run a full set of perf tests with the fsmonitor feature disabled.\n #\n \n+trace_start fsmonitor-disabled\n test_expect_success \"setup without fsmonitor\" '\n \tunset INTEGRATION_SCRIPT &&\n \tgit config --unset core.fsmonitor &&\n@@ -243,5 +274,6 @@ test_expect_success \"setup without fsmonitor\" '\n '\n \n test_fsmonitor_suite\n+trace_stop\n \n test_done\n-- \ngitgitgadget\n\n"},{"id":"416047","messageId":"e570f7316cce4d3cd6cedbb551124b7b317f11d3.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 01/11] p7519: do not rely on \"xargs -d\" in test","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:40Z","receivedAt":"2021-02-03T15:38:51Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nConvert the test to use a more portable method to update the mtime on a\nlarge number of files under version control.\n\nThe Mac version of xargs does not support the \"-d\" option.\nLikewise, the \"-0\" and \"--null\" options are not portable.\n\nFurthermore, use `test-tool chmtime` rather than `touch` to update the\nmtime to ensure that it is actually updated (especially on file systems\nwith only whole second resolution).\n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/p7519-fsmonitor.sh | 12 +++++++++++-\n 1 file changed, 11 insertions(+), 1 deletion(-)\n\ndiff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\nindex 9b43342806b..6677e0ef7ab 100755\n--- a/t/perf/p7519-fsmonitor.sh\n+++ b/t/perf/p7519-fsmonitor.sh\n@@ -164,8 +164,18 @@ test_fsmonitor_suite() {\n \t\tgit status -uall\n \t'\n \n+\t# Update the mtimes on upto 100k files to make status think\n+\t# that they are dirty. For simplicity, omit any files with\n+\t# LFs (i.e. anything that ls-files thinks it needs to dquote).\n+\t# Then fully backslash-quote the paths to capture any\n+\t# whitespace so that they pass thru xargs properly.\n+\t#\n \ttest_perf_w_drop_caches \"status (dirty) ($DESC)\" '\n-\t\tgit ls-files | head -100000 | xargs -d \"\\n\" touch -h &&\n+\t\tgit ls-files | \\\n+\t\t\thead -100000 | \\\n+\t\t\tgrep -v \\\" | \\\n+\t\t\tsed '\\''s/\\(.\\)/\\\\\\1/g'\\'' | \\\n+\t\t\txargs test-tool chmtime -300 &&\n \t\tgit status\n \t'\n \n-- \ngitgitgadget\n\n"},{"id":"416048","messageId":"3042fc92fe6d5ff05fd2e4ac96ef936a69b2a2ff.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 02/11] p7519: fix watchman watch-list test on Windows","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:41Z","receivedAt":"2021-02-03T15:39:26Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nOnly use the final portion of the test trash directory file name\nwhen verifying that Watchman was started.\n\nOn Windows and under the SDK, $GIT_WORKTREE is a cygwin-style\npath with forward slashes and a \"/c/\" drive name. However\n`watchman watch-list` reports a proper Windows-style pathname\nwith drive letters and backslashes. This causes the grep to\nfail. Since we don't really care about the full pathname (and\nwe really don't want to bother with normalizaing them), just see\nif the test-name portion of the path is found.\n\nSigned-off-by: Jeff Hostetler \n---\n t/perf/p7519-fsmonitor.sh | 2 +-\n 1 file changed, 1 insertion(+), 1 deletion(-)\n\ndiff --git a/t/perf/p7519-fsmonitor.sh b/t/perf/p7519-fsmonitor.sh\nindex 6677e0ef7ab..21d525541d5 100755\n--- a/t/perf/p7519-fsmonitor.sh\n+++ b/t/perf/p7519-fsmonitor.sh\n@@ -101,7 +101,7 @@ test_expect_success \"one time repo setup\" '\n \t# If Watchman exists, watch the work tree and attempt a query.\n \tif test_have_prereq WATCHMAN; then\n \t\twatchman watch \"$GIT_WORK_TREE\" &&\n-\t\twatchman watch-list | grep -q -F \"$GIT_WORK_TREE\"\n+\t\twatchman watch-list | grep -q -F \"p7519-fsmonitor\"\n \tfi\n '\n \n-- \ngitgitgadget\n\n"},{"id":"416049","messageId":"d150a2d4576e1519e3a4b228a30ad3c3e63fa150.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 06/11] read-cache: log the number of lstat calls to trace2","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:45Z","receivedAt":"2021-02-03T15:40:05Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nReport the total number of calls made to lstat() inside of refresh_index().\n\nFSMonitor improves the performance of commands like `git status` by\navoiding scanning the disk for changed files. This can be seen in\n`refresh_index()`. Let's measure this.\n\nSigned-off-by: Jeff Hostetler \n---\n read-cache.c | 17 ++++++++++++++---\n 1 file changed, 14 insertions(+), 3 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex ecf6f689940..893cc41e1d9 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -1364,7 +1364,8 @@ int add_index_entry(struct index_state *istate, struct cache_entry *ce, int opti\n static struct cache_entry *refresh_cache_ent(struct index_state *istate,\n \t\t\t\t\t struct cache_entry *ce,\n \t\t\t\t\t unsigned int options, int *err,\n-\t\t\t\t\t int *changed_ret)\n+\t\t\t\t\t int *changed_ret,\n+\t\t\t\t\t int *t2_did_lstat)\n {\n \tstruct stat st;\n \tstruct cache_entry *updated;\n@@ -1406,6 +1407,8 @@ static struct cache_entry *refresh_cache_ent(struct index_state *istate,\n \t\treturn NULL;\n \t}\n \n+\tif (t2_did_lstat)\n+\t\t*t2_did_lstat = 1;\n \tif (lstat(ce->name, &st) < 0) {\n \t\tif (ignore_missing && errno == ENOENT)\n \t\t\treturn ce;\n@@ -1519,6 +1522,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \tconst char *added_fmt;\n \tconst char *unmerged_fmt;\n \tstruct progress *progress = NULL;\n+\tint t2_sum_lstat = 0;\n \n \tif (flags & REFRESH_PROGRESS && isatty(2))\n \t\tprogress = start_delayed_progress(_(\"Refresh index\"),\n@@ -1536,11 +1540,13 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \t * we only have to do the special cases that are left.\n \t */\n \tpreload_index(istate, pathspec, 0);\n+\ttrace2_region_enter(\"index\", \"refresh\", NULL);\n \tfor (i = 0; i < istate->cache_nr; i++) {\n \t\tstruct cache_entry *ce, *new_entry;\n \t\tint cache_errno = 0;\n \t\tint changed = 0;\n \t\tint filtered = 0;\n+\t\tint t2_did_lstat = 0;\n \n \t\tce = istate->cache[i];\n \t\tif (ignore_submodules && S_ISGITLINK(ce->ce_mode))\n@@ -1566,7 +1572,10 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \t\tif (filtered)\n \t\t\tcontinue;\n \n-\t\tnew_entry = refresh_cache_ent(istate, ce, options, &cache_errno, &changed);\n+\t\tnew_entry = refresh_cache_ent(istate, ce, options,\n+\t\t\t\t\t &cache_errno, &changed,\n+\t\t\t\t\t &t2_did_lstat);\n+\t\tt2_sum_lstat += t2_did_lstat;\n \t\tif (new_entry == ce)\n \t\t\tcontinue;\n \t\tif (progress)\n@@ -1602,6 +1611,8 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \n \t\treplace_index_entry(istate, i, new_entry);\n \t}\n+\ttrace2_data_intmax(\"index\", NULL, \"refresh/sum_lstat\", t2_sum_lstat);\n+\ttrace2_region_leave(\"index\", \"refresh\", NULL);\n \tif (progress) {\n \t\tdisplay_progress(progress, istate->cache_nr);\n \t\tstop_progress(&progress);\n@@ -1614,7 +1625,7 @@ struct cache_entry *refresh_cache_entry(struct index_state *istate,\n \t\t\t\t\tstruct cache_entry *ce,\n \t\t\t\t\tunsigned int options)\n {\n-\treturn refresh_cache_ent(istate, ce, options, NULL, NULL);\n+\treturn refresh_cache_ent(istate, ce, options, NULL, NULL, NULL);\n }\n \n \n-- \ngitgitgadget\n\n"},{"id":"416050","messageId":"33cc0b838fadcfd8fc9cf769d77b83f3775e363f.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 07/11] read-cache: log the number of scanned files to trace2","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:46Z","receivedAt":"2021-02-03T15:40:09Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nReport the number of files in the working directory that were read and\ntheir hashes verified in `refresh_index()`.\n\nFSMonitor improves the performance of commands like `git status` by\navoiding scanning the disk for changed files. Let's measure this.\n\nSigned-off-by: Jeff Hostetler \n---\n read-cache.c | 13 ++++++++++---\n 1 file changed, 10 insertions(+), 3 deletions(-)\n\ndiff --git a/read-cache.c b/read-cache.c\nindex 893cc41e1d9..c9dd7f4015e 100644\n--- a/read-cache.c\n+++ b/read-cache.c\n@@ -1365,7 +1365,8 @@ static struct cache_entry *refresh_cache_ent(struct index_state *istate,\n \t\t\t\t\t struct cache_entry *ce,\n \t\t\t\t\t unsigned int options, int *err,\n \t\t\t\t\t int *changed_ret,\n-\t\t\t\t\t int *t2_did_lstat)\n+\t\t\t\t\t int *t2_did_lstat,\n+\t\t\t\t\t int *t2_did_scan)\n {\n \tstruct stat st;\n \tstruct cache_entry *updated;\n@@ -1445,6 +1446,8 @@ static struct cache_entry *refresh_cache_ent(struct index_state *istate,\n \t\t}\n \t}\n \n+\tif (t2_did_scan)\n+\t\t*t2_did_scan = 1;\n \tif (ie_modified(istate, ce, &st, options)) {\n \t\tif (err)\n \t\t\t*err = EINVAL;\n@@ -1523,6 +1526,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \tconst char *unmerged_fmt;\n \tstruct progress *progress = NULL;\n \tint t2_sum_lstat = 0;\n+\tint t2_sum_scan = 0;\n \n \tif (flags & REFRESH_PROGRESS && isatty(2))\n \t\tprogress = start_delayed_progress(_(\"Refresh index\"),\n@@ -1547,6 +1551,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \t\tint changed = 0;\n \t\tint filtered = 0;\n \t\tint t2_did_lstat = 0;\n+\t\tint t2_did_scan = 0;\n \n \t\tce = istate->cache[i];\n \t\tif (ignore_submodules && S_ISGITLINK(ce->ce_mode))\n@@ -1574,8 +1579,9 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \n \t\tnew_entry = refresh_cache_ent(istate, ce, options,\n \t\t\t\t\t &cache_errno, &changed,\n-\t\t\t\t\t &t2_did_lstat);\n+\t\t\t\t\t &t2_did_lstat, &t2_did_scan);\n \t\tt2_sum_lstat += t2_did_lstat;\n+\t\tt2_sum_scan += t2_did_scan;\n \t\tif (new_entry == ce)\n \t\t\tcontinue;\n \t\tif (progress)\n@@ -1612,6 +1618,7 @@ int refresh_index(struct index_state *istate, unsigned int flags,\n \t\treplace_index_entry(istate, i, new_entry);\n \t}\n \ttrace2_data_intmax(\"index\", NULL, \"refresh/sum_lstat\", t2_sum_lstat);\n+\ttrace2_data_intmax(\"index\", NULL, \"refresh/sum_scan\", t2_sum_scan);\n \ttrace2_region_leave(\"index\", \"refresh\", NULL);\n \tif (progress) {\n \t\tdisplay_progress(progress, istate->cache_nr);\n@@ -1625,7 +1632,7 @@ struct cache_entry *refresh_cache_entry(struct index_state *istate,\n \t\t\t\t\tstruct cache_entry *ce,\n \t\t\t\t\tunsigned int options)\n {\n-\treturn refresh_cache_ent(istate, ce, options, NULL, NULL, NULL);\n+\treturn refresh_cache_ent(istate, ce, options, NULL, NULL, NULL, NULL);\n }\n \n \n-- \ngitgitgadget\n\n"},{"id":"416051","messageId":"5410d3ab61d1c9bb2a4d737bca8a81625e333da0.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 11/11] fsmonitor: refactor initialization of fsmonitor_last_update token","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:50Z","receivedAt":"2021-02-03T15:40:27Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nIsolate and document initialization of `istate->fsmonitor_last_update`.\nThis field should contain a fsmonitor-specific opaque token, but we\nneed to initialize it before we can actually talk to a fsmonitor process,\nso we create a generic default value.\n\nSigned-off-by: Jeff Hostetler \n---\n fsmonitor.c | 35 ++++++++++++++++++++++++++++++++---\n 1 file changed, 32 insertions(+), 3 deletions(-)\n\ndiff --git a/fsmonitor.c b/fsmonitor.c\nindex 64deeda597e..e12214b3007 100644\n--- a/fsmonitor.c\n+++ b/fsmonitor.c\n@@ -343,16 +343,45 @@ void refresh_fsmonitor(struct index_state *istate)\n \tistate->fsmonitor_last_update = strbuf_detach(&last_update_token, NULL);\n }\n \n+/*\n+ * The caller wants to turn on FSMonitor. And when the caller writes\n+ * the index to disk, a FSMonitor extension should be included. This\n+ * requires that `istate->fsmonitor_last_update` not be NULL. But we\n+ * have not actually talked to a FSMonitor process yet, so we don't\n+ * have an initial value for this field.\n+ *\n+ * For a protocol V1 FSMonitor process, this field is a formatted\n+ * \"nanoseconds since epoch\" field. However, for a protocol V2\n+ * FSMonitor process, this field is an opaque token.\n+ *\n+ * Historically, `add_fsmonitor()` has initialized this field to the\n+ * current time for protocol V1 processes. There are lots of race\n+ * conditions here, but that code has shipped...\n+ *\n+ * The only true solution is to use a V2 FSMonitor and get a current\n+ * or default token value (that it understands), but we cannot do that\n+ * until we have actually talked to an instance of the FSMonitor process\n+ * (but the protocol requires that we send a token first...).\n+ *\n+ * For simplicity, just initialize like we have a V1 process and require\n+ * that V2 processes adapt.\n+ */\n+static void initialize_fsmonitor_last_update(struct index_state *istate)\n+{\n+\tstruct strbuf last_update = STRBUF_INIT;\n+\n+\tstrbuf_addf(&last_update, \"%\"PRIu64\"\", getnanotime());\n+\tistate->fsmonitor_last_update = strbuf_detach(&last_update, NULL);\n+}\n+\n void add_fsmonitor(struct index_state *istate)\n {\n \tunsigned int i;\n-\tstruct strbuf last_update = STRBUF_INIT;\n \n \tif (!istate->fsmonitor_last_update) {\n \t\ttrace_printf_key(&trace_fsmonitor, \"add fsmonitor\");\n \t\tistate->cache_changed |= FSMONITOR_CHANGED;\n-\t\tstrbuf_addf(&last_update, \"%\"PRIu64\"\", getnanotime());\n-\t\tistate->fsmonitor_last_update = strbuf_detach(&last_update, NULL);\n+\t\tinitialize_fsmonitor_last_update(istate);\n \n \t\t/* reset the fsmonitor state */\n \t\tfor (i = 0; i < istate->cache_nr; i++)\n-- \ngitgitgadget\n"},{"id":"416052","messageId":"6ec4a4468f6a068291c2ad6b1ddb75f68c13c3a8.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 09/11] fsmonitor: log FSMN token when reading and writing the index","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:48Z","receivedAt":"2021-02-03T15:40:35Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nSigned-off-by: Jeff Hostetler \n---\n fsmonitor.c | 12 ++++++++++--\n 1 file changed, 10 insertions(+), 2 deletions(-)\n\ndiff --git a/fsmonitor.c b/fsmonitor.c\nindex 7a2be24cd43..3105dc370ab 100644\n--- a/fsmonitor.c\n+++ b/fsmonitor.c\n@@ -87,7 +87,11 @@ int read_fsmonitor_extension(struct index_state *istate, const void *data,\n \t\tBUG(\"fsmonitor_dirty has more entries than the index (%\"PRIuMAX\" > %u)\",\n \t\t (uintmax_t)istate->fsmonitor_dirty->bit_size, istate->cache_nr);\n \n-\ttrace_printf_key(&trace_fsmonitor, \"read fsmonitor extension successful\");\n+\ttrace2_data_string(\"index\", NULL, \"extension/fsmn/read/token\",\n+\t\t\t istate->fsmonitor_last_update);\n+\ttrace_printf_key(&trace_fsmonitor,\n+\t\t\t \"read fsmonitor extension successful '%s'\",\n+\t\t\t istate->fsmonitor_last_update);\n \treturn 0;\n }\n \n@@ -133,7 +137,11 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate)\n \tput_be32(&ewah_size, sb->len - ewah_start);\n \tmemcpy(sb->buf + fixup, &ewah_size, sizeof(uint32_t));\n \n-\ttrace_printf_key(&trace_fsmonitor, \"write fsmonitor extension successful\");\n+\ttrace2_data_string(\"index\", NULL, \"extension/fsmn/write/token\",\n+\t\t\t istate->fsmonitor_last_update);\n+\ttrace_printf_key(&trace_fsmonitor,\n+\t\t\t \"write fsmonitor extension successful '%s'\",\n+\t\t\t istate->fsmonitor_last_update);\n }\n \n /*\n-- \ngitgitgadget\n\n"},{"id":"416053","messageId":"2ac66f07a59abe84c0105d5d97560fc3b22a9955.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 10/11] fsmonitor: allow all entries for a folder to be invalidated","fromName":"Kevin Willford via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:49Z","receivedAt":"2021-02-03T15:40:49Z","isPatch":true,"sender":{"key":"kewillf@microsoft.com","avatar":null},"body":"From: Kevin Willford \n\nAllow fsmonitor to report directory changes by reporting paths with a\ntrailing slash.\n\nSigned-off-by: Jeff Hostetler \nSigned-off-by: Kevin Willford \nSigned-off-by: Johannes Schindelin \n---\n fsmonitor.c | 31 ++++++++++++++++++++++++++-----\n 1 file changed, 26 insertions(+), 5 deletions(-)\n\ndiff --git a/fsmonitor.c b/fsmonitor.c\nindex 3105dc370ab..64deeda597e 100644\n--- a/fsmonitor.c\n+++ b/fsmonitor.c\n@@ -190,13 +190,34 @@ int fsmonitor_is_trivial_response(const struct strbuf *query_result)\n \treturn is_trivial;\n }\n \n-static void fsmonitor_refresh_callback(struct index_state *istate, const char *name)\n+static void fsmonitor_refresh_callback(struct index_state *istate, char *name)\n {\n-\tint pos = index_name_pos(istate, name, strlen(name));\n+\tint i, len = strlen(name);\n+\tif (name[len - 1] == '/') {\n+\n+\t\t/*\n+\t\t * TODO We should binary search to find the first path with\n+\t\t * TODO this directory prefix. Then linearly update entries\n+\t\t * TODO while the prefix matches. Taking care to search without\n+\t\t * TODO the trailing slash -- because '/' sorts after a few\n+\t\t * TODO interesting special chars, like '.' and ' '.\n+\t\t */\n+\n+\t\t/* Mark all entries for the folder invalid */\n+\t\tfor (i = 0; i < istate->cache_nr; i++) {\n+\t\t\tif (istate->cache[i]->ce_flags & CE_FSMONITOR_VALID &&\n+\t\t\t starts_with(istate->cache[i]->name, name))\n+\t\t\t\tistate->cache[i]->ce_flags &= ~CE_FSMONITOR_VALID;\n+\t\t}\n+\t\t/* Need to remove the / from the path for the untracked cache */\n+\t\tname[len - 1] = '\\0';\n+\t} else {\n+\t\tint pos = index_name_pos(istate, name, strlen(name));\n \n-\tif (pos >= 0) {\n-\t\tstruct cache_entry *ce = istate->cache[pos];\n-\t\tce->ce_flags &= ~CE_FSMONITOR_VALID;\n+\t\tif (pos >= 0) {\n+\t\t\tstruct cache_entry *ce = istate->cache[pos];\n+\t\t\tce->ce_flags &= ~CE_FSMONITOR_VALID;\n+\t\t}\n \t}\n \n \t/*\n-- \ngitgitgadget\n\n"},{"id":"416054","messageId":"c043bccc8affd56f920158935671f9e172d6d343.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 08/11] fsmonitor: log invocation of FSMonitor hook to trace2","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:47Z","receivedAt":"2021-02-03T15:40:54Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nLet's measure the time taken to request and receive FSMonitor data\nvia the hook API and the size of the response.\n\nSigned-off-by: Jeff Hostetler \n---\n fsmonitor.c | 29 ++++++++++++++++++++++++++++-\n fsmonitor.h | 5 +++++\n 2 files changed, 33 insertions(+), 1 deletion(-)\n\ndiff --git a/fsmonitor.c b/fsmonitor.c\nindex ca031c3abb8..7a2be24cd43 100644\n--- a/fsmonitor.c\n+++ b/fsmonitor.c\n@@ -142,6 +142,7 @@ void write_fsmonitor_extension(struct strbuf *sb, struct index_state *istate)\n static int query_fsmonitor(int version, const char *last_update, struct strbuf *query_result)\n {\n \tstruct child_process cp = CHILD_PROCESS_INIT;\n+\tint result;\n \n \tif (!core_fsmonitor)\n \t\treturn -1;\n@@ -152,7 +153,33 @@ static int query_fsmonitor(int version, const char *last_update, struct strbuf *\n \tcp.use_shell = 1;\n \tcp.dir = get_git_work_tree();\n \n-\treturn capture_command(&cp, query_result, 1024);\n+\ttrace2_region_enter(\"fsm_hook\", \"query\", NULL);\n+\n+\tresult = capture_command(&cp, query_result, 1024);\n+\n+\tif (result)\n+\t\ttrace2_data_intmax(\"fsm_hook\", NULL, \"query/failed\", result);\n+\telse {\n+\t\ttrace2_data_intmax(\"fsm_hook\", NULL, \"query/response-length\",\n+\t\t\t\t query_result->len);\n+\n+\t\tif (fsmonitor_is_trivial_response(query_result))\n+\t\t\ttrace2_data_intmax(\"fsm_hook\", NULL,\n+\t\t\t\t\t \"query/trivial-response\", 1);\n+\t}\n+\n+\ttrace2_region_leave(\"fsm_hook\", \"query\", NULL);\n+\n+\treturn result;\n+}\n+\n+int fsmonitor_is_trivial_response(const struct strbuf *query_result)\n+{\n+\tstatic char trivial_response[3] = { '\\0', '/', '\\0' };\n+\tint is_trivial = !memcmp(trivial_response,\n+\t\t\t\t &query_result->buf[query_result->len - 3], 3);\n+\n+\treturn is_trivial;\n }\n \n static void fsmonitor_refresh_callback(struct index_state *istate, const char *name)\ndiff --git a/fsmonitor.h b/fsmonitor.h\nindex 739318ab6d1..7f1794b90b0 100644\n--- a/fsmonitor.h\n+++ b/fsmonitor.h\n@@ -44,6 +44,11 @@ void tweak_fsmonitor(struct index_state *istate);\n */\n void refresh_fsmonitor(struct index_state *istate);\n \n+/*\n+ * Does the received result contain the \"trivial\" response?\n+ */\n+int fsmonitor_is_trivial_response(const struct strbuf *query_result);\n+\n /*\n * Set the given cache entries CE_FSMONITOR_VALID bit. This should be\n * called any time the cache entry has been updated to reflect the\n-- \ngitgitgadget\n\n"},{"id":"416055","messageId":"3c5035e4649d18d96fc0206c2ceacc6ffe99deff.1612366490.git.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"[PATCH v2 05/11] preload-index: log the number of lstat calls to trace2","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:44Z","receivedAt":"2021-02-03T15:52:47Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"From: Jeff Hostetler \n\nReport the total number of calls made to lstat() inside preload_index().\n\nFSMonitor improves the performance of commands like `git status` by\navoiding scanning the disk for changed files. This can be seen in\n`preload_index()`. Let's measure this.\n\nSigned-off-by: Jeff Hostetler \n---\n preload-index.c | 10 ++++++++++\n 1 file changed, 10 insertions(+)\n\ndiff --git a/preload-index.c b/preload-index.c\nindex ed6eaa47388..e5529a58636 100644\n--- a/preload-index.c\n+++ b/preload-index.c\n@@ -31,6 +31,7 @@ struct thread_data {\n \tstruct pathspec pathspec;\n \tstruct progress_data *progress;\n \tint offset, nr;\n+\tint t2_nr_lstat;\n };\n \n static void *preload_thread(void *_data)\n@@ -73,6 +74,7 @@ static void *preload_thread(void *_data)\n \t\t\tcontinue;\n \t\tif (threaded_has_symlink_leading_path(&cache, ce->name, ce_namelen(ce)))\n \t\t\tcontinue;\n+\t\tp->t2_nr_lstat++;\n \t\tif (lstat(ce->name, &st))\n \t\t\tcontinue;\n \t\tif (ie_match_stat(index, ce, &st, CE_MATCH_RACY_IS_DIRTY|CE_MATCH_IGNORE_FSMONITOR))\n@@ -98,6 +100,7 @@ void preload_index(struct index_state *index,\n \tint threads, i, work, offset;\n \tstruct thread_data data[MAX_PARALLEL];\n \tstruct progress_data pd;\n+\tint t2_sum_lstat = 0;\n \n \tif (!HAVE_THREADS || !core_preload_index)\n \t\treturn;\n@@ -107,6 +110,9 @@ void preload_index(struct index_state *index,\n \t\tthreads = 2;\n \tif (threads < 2)\n \t\treturn;\n+\n+\ttrace2_region_enter(\"index\", \"preload\", NULL);\n+\n \ttrace_performance_enter();\n \tif (threads > MAX_PARALLEL)\n \t\tthreads = MAX_PARALLEL;\n@@ -141,10 +147,14 @@ void preload_index(struct index_state *index,\n \t\tstruct thread_data *p = data+i;\n \t\tif (pthread_join(p->pthread, NULL))\n \t\t\tdie(\"unable to join threaded lstat\");\n+\t\tt2_sum_lstat += p->t2_nr_lstat;\n \t}\n \tstop_progress(&pd.progress);\n \n \ttrace_performance_leave(\"preload index\");\n+\n+\ttrace2_data_intmax(\"index\", NULL, \"preload/sum_lstat\", t2_sum_lstat);\n+\ttrace2_region_leave(\"index\", \"preload\", NULL);\n }\n \n int repo_read_index_preload(struct repository *repo,\n-- \ngitgitgadget\n\n"},{"id":"416057","messageId":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"[PATCH v2 00/11] FSMonitor Preliminary Commits","fromName":"Jeff Hostetler via GitGitGadget","fromEmail":"gitgitgadget@gmail.com","sentAt":"2021-02-03T15:34:39Z","receivedAt":"2021-02-03T15:54:54Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"Here is version 2 of this series.\n\nIn version 1, I replaced the non-portable \"xargs -d\" with \"xargs -0\", but it\nturns out that that too is not universally available. In this version I\nreplace the need for either one by filtering out the problematic paths (such\nas ones with LFs) and quoting paths to handle whitespace. The resulting\npaths can be passed to xargs without any arguments.\n\nAlso, I updated the test to use test-tool chmtime rather than touch to\nensure that the files actually look dirty on low-resolution file systems.\n\nJeff Hostetler (10):\n p7519: do not rely on \"xargs -d\" in test\n p7519: fix watchman watch-list test on Windows\n p7519: move watchman cleanup earlier in the test\n p7519: add trace logging during perf test\n preload-index: log the number of lstat calls to trace2\n read-cache: log the number of lstat calls to trace2\n read-cache: log the number of scanned files to trace2\n fsmonitor: log invocation of FSMonitor hook to trace2\n fsmonitor: log FSMN token when reading and writing the index\n fsmonitor: refactor initialization of fsmonitor_last_update token\n\nKevin Willford (1):\n fsmonitor: allow all entries for a folder to be invalidated\n\n fsmonitor.c | 107 ++++++++++++++++++++++++++++++++++----\n fsmonitor.h | 5 ++\n preload-index.c | 10 ++++\n read-cache.c | 24 +++++++--\n t/perf/.gitignore | 1 +\n t/perf/Makefile | 4 +-\n t/perf/p7519-fsmonitor.sh | 71 +++++++++++++++++++++----\n 7 files changed, 196 insertions(+), 26 deletions(-)\n\n\nbase-commit: 71ca53e8125e36efbda17293c50027d31681a41f\nPublished-As: https://github.com/gitgitgadget/git/releases/tag/pr-860%2Fjeffhostetler%2Ffsmonitor-prework-v2\nFetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-860/jeffhostetler/fsmonitor-prework-v2\nPull-Request: https://github.com/gitgitgadget/git/pull/860\n\nRange-diff vs v1:\n\n 1: cf252e24b8c ! 1: e570f7316cc p7519: use xargs -0 rather than -d in test\n @@ Metadata\n Author: Jeff Hostetler \n \n ## Commit message ##\n - p7519: use xargs -0 rather than -d in test\n + p7519: do not rely on \"xargs -d\" in test\n \n - The Mac version of xargs does not support the \"-d\" option. Convert the test\n - setup to pipe the data set thru `lf_to_nul | xargs -0` instead.\n + Convert the test to use a more portable method to update the mtime on a\n + large number of files under version control.\n +\n + The Mac version of xargs does not support the \"-d\" option.\n + Likewise, the \"-0\" and \"--null\" options are not portable.\n +\n + Furthermore, use `test-tool chmtime` rather than `touch` to update the\n + mtime to ensure that it is actually updated (especially on file systems\n + with only whole second resolution).\n \n Signed-off-by: Jeff Hostetler \n \n ## t/perf/p7519-fsmonitor.sh ##\n @@ t/perf/p7519-fsmonitor.sh: test_fsmonitor_suite() {\n + \t\tgit status -uall\n \t'\n \n ++\t# Update the mtimes on upto 100k files to make status think\n ++\t# that they are dirty. For simplicity, omit any files with\n ++\t# LFs (i.e. anything that ls-files thinks it needs to dquote).\n ++\t# Then fully backslash-quote the paths to capture any\n ++\t# whitespace so that they pass thru xargs properly.\n ++\t#\n \ttest_perf_w_drop_caches \"status (dirty) ($DESC)\" '\n -\t\tgit ls-files | head -100000 | xargs -d \"\\n\" touch -h &&\n -+\t\tgit ls-files | head -100000 | lf_to_nul | xargs -0 touch -h &&\n ++\t\tgit ls-files | \\\n ++\t\t\thead -100000 | \\\n ++\t\t\tgrep -v \\\" | \\\n ++\t\t\tsed '\\''s/\\(.\\)/\\\\\\1/g'\\'' | \\\n ++\t\t\txargs test-tool chmtime -300 &&\n \t\tgit status\n \t'\n \n 2: a641f9e357c = 2: 3042fc92fe6 p7519: fix watchman watch-list test on Windows\n 3: 2af6858716f = 3: 9ceba5e6942 p7519: move watchman cleanup earlier in the test\n 4: 8de9985a706 = 4: f6ea0a51f50 p7519: add trace logging during perf test\n 5: cdd49f1fdb1 = 5: 3c5035e4649 preload-index: log the number of lstat calls to trace2\n 6: 65488f7a1bf = 6: d150a2d4576 read-cache: log the number of lstat calls to trace2\n 7: c84531f6244 = 7: 33cc0b838fa read-cache: log the number of scanned files to trace2\n 8: ef64b60c7a0 = 8: c043bccc8af fsmonitor: log invocation of FSMonitor hook to trace2\n 9: edb88ffe39e = 9: 6ec4a4468f6 fsmonitor: log FSMN token when reading and writing the index\n 10: 384d2eff863 = 10: 2ac66f07a59 fsmonitor: allow all entries for a folder to be invalidated\n 11: 4686196bbc6 = 11: 5410d3ab61d fsmonitor: refactor initialization of fsmonitor_last_update token\n\n-- \ngitgitgadget\n"},{"id":"416094","messageId":"YBsTcDokmFcL8N1o@nand.local","threadId":"55087","inReplyTo":"pull.860.git.1612216941.gitgitgadget@gmail.com","subject":"Re: [PATCH 00/11] FSMonitor Preliminary Commits","fromName":"Taylor Blau","fromEmail":"me@ttaylorr.com","sentAt":"2021-02-03T21:19:44Z","receivedAt":"2021-02-03T21:20:47Z","isPatch":true,"sender":{"key":"me@ttaylorr.com","avatar":"https://avatars.githubusercontent.com/u/301000140?v=4"},"body":"Hi Jeff,\n\nOn Mon, Feb 01, 2021 at 10:02:09PM +0000, Jeff Hostetler via GitGitGadget wrote:\n> This patch series fixes runtime errors in t/perf/p7519 on Windows and MacOS.\n> It adds Trace2 logging to p7519 to make it easier to compare results when\n> Watchman is enabled and disabled. And finally, it adds some Trace2 regions\n> and data events around our usage of Watchman and the existing FSMonitor\n> framework.\n\nI read v2 of this series and it all looked very sane to me. I didn't\nhave much in the way of comments or concerns along the way, so that\nversion of the series has my reviewed-by.\n\nThanks,\nTaylor\n"},{"id":"417093","messageId":"42714a3b-cccc-d61b-1dc1-5f2f8fcaa0dd@jeffhostetler.com","threadId":"55087","inReplyTo":"pull.860.v2.git.1612366490.gitgitgadget@gmail.com","subject":"Re: [PATCH v2 00/11] FSMonitor Preliminary Commits","fromName":"Jeff Hostetler","fromEmail":"git@jeffhostetler.com","sentAt":"2021-02-16T19:00:05Z","receivedAt":"2021-02-16T19:00:50Z","isPatch":true,"sender":{"key":"git@jeffhostetler.com","avatar":null},"body":"\n\nOn 2/3/21 10:34 AM, Jeff Hostetler via GitGitGadget wrote:\n> Here is version 2 of this series.\n\nI didn't see this series in the \"what's cooking\" emails and\nwas wondering if there was something that I still needed to\nattend to.\n\nThanks\nJeff\n\n\n> \n> In version 1, I replaced the non-portable \"xargs -d\" with \"xargs -0\", but it\n> turns out that that too is not universally available. In this version I\n> replace the need for either one by filtering out the problematic paths (such\n> as ones with LFs) and quoting paths to handle whitespace. The resulting\n> paths can be passed to xargs without any arguments.\n> \n> Also, I updated the test to use test-tool chmtime rather than touch to\n> ensure that the files actually look dirty on low-resolution file systems.\n> \n> Jeff Hostetler (10):\n> p7519: do not rely on \"xargs -d\" in test\n> p7519: fix watchman watch-list test on Windows\n> p7519: move watchman cleanup earlier in the test\n> p7519: add trace logging during perf test\n> preload-index: log the number of lstat calls to trace2\n> read-cache: log the number of lstat calls to trace2\n> read-cache: log the number of scanned files to trace2\n> fsmonitor: log invocation of FSMonitor hook to trace2\n> fsmonitor: log FSMN token when reading and writing the index\n> fsmonitor: refactor initialization of fsmonitor_last_update token\n> \n> Kevin Willford (1):\n> fsmonitor: allow all entries for a folder to be invalidated\n> \n> fsmonitor.c | 107 ++++++++++++++++++++++++++++++++++----\n> fsmonitor.h | 5 ++\n> preload-index.c | 10 ++++\n> read-cache.c | 24 +++++++--\n> t/perf/.gitignore | 1 +\n> t/perf/Makefile | 4 +-\n> t/perf/p7519-fsmonitor.sh | 71 +++++++++++++++++++++----\n> 7 files changed, 196 insertions(+), 26 deletions(-)\n> \n> \n> base-commit: 71ca53e8125e36efbda17293c50027d31681a41f\n> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-860%2Fjeffhostetler%2Ffsmonitor-prework-v2\n> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-860/jeffhostetler/fsmonitor-prework-v2\n> Pull-Request: https://github.com/gitgitgadget/git/pull/860\n> \n> Range-diff vs v1:\n> \n> 1: cf252e24b8c ! 1: e570f7316cc p7519: use xargs -0 rather than -d in test\n> @@ Metadata\n> Author: Jeff Hostetler \n> \n> ## Commit message ##\n> - p7519: use xargs -0 rather than -d in test\n> + p7519: do not rely on \"xargs -d\" in test\n> \n> - The Mac version of xargs does not support the \"-d\" option. Convert the test\n> - setup to pipe the data set thru `lf_to_nul | xargs -0` instead.\n> + Convert the test to use a more portable method to update the mtime on a\n> + large number of files under version control.\n> +\n> + The Mac version of xargs does not support the \"-d\" option.\n> + Likewise, the \"-0\" and \"--null\" options are not portable.\n> +\n> + Furthermore, use `test-tool chmtime` rather than `touch` to update the\n> + mtime to ensure that it is actually updated (especially on file systems\n> + with only whole second resolution).\n> \n> Signed-off-by: Jeff Hostetler \n> \n> ## t/perf/p7519-fsmonitor.sh ##\n> @@ t/perf/p7519-fsmonitor.sh: test_fsmonitor_suite() {\n> + \t\tgit status -uall\n> \t'\n> \n> ++\t# Update the mtimes on upto 100k files to make status think\n> ++\t# that they are dirty. For simplicity, omit any files with\n> ++\t# LFs (i.e. anything that ls-files thinks it needs to dquote).\n> ++\t# Then fully backslash-quote the paths to capture any\n> ++\t# whitespace so that they pass thru xargs properly.\n> ++\t#\n> \ttest_perf_w_drop_caches \"status (dirty) ($DESC)\" '\n> -\t\tgit ls-files | head -100000 | xargs -d \"\\n\" touch -h &&\n> -+\t\tgit ls-files | head -100000 | lf_to_nul | xargs -0 touch -h &&\n> ++\t\tgit ls-files | \\\n> ++\t\t\thead -100000 | \\\n> ++\t\t\tgrep -v \\\" | \\\n> ++\t\t\tsed '\\''s/\\(.\\)/\\\\\\1/g'\\'' | \\\n> ++\t\t\txargs test-tool chmtime -300 &&\n> \t\tgit status\n> \t'\n> \n> 2: a641f9e357c = 2: 3042fc92fe6 p7519: fix watchman watch-list test on Windows\n> 3: 2af6858716f = 3: 9ceba5e6942 p7519: move watchman cleanup earlier in the test\n> 4: 8de9985a706 = 4: f6ea0a51f50 p7519: add trace logging during perf test\n> 5: cdd49f1fdb1 = 5: 3c5035e4649 preload-index: log the number of lstat calls to trace2\n> 6: 65488f7a1bf = 6: d150a2d4576 read-cache: log the number of lstat calls to trace2\n> 7: c84531f6244 = 7: 33cc0b838fa read-cache: log the number of scanned files to trace2\n> 8: ef64b60c7a0 = 8: c043bccc8af fsmonitor: log invocation of FSMonitor hook to trace2\n> 9: edb88ffe39e = 9: 6ec4a4468f6 fsmonitor: log FSMN token when reading and writing the index\n> 10: 384d2eff863 = 10: 2ac66f07a59 fsmonitor: allow all entries for a folder to be invalidated\n> 11: 4686196bbc6 = 11: 5410d3ab61d fsmonitor: refactor initialization of fsmonitor_last_update token\n> \n"},{"id":"417136","messageId":"xmqqmtw3mpbz.fsf@gitster.g","threadId":"55087","inReplyTo":"42714a3b-cccc-d61b-1dc1-5f2f8fcaa0dd@jeffhostetler.com","subject":"Re: [PATCH v2 00/11] FSMonitor Preliminary Commits","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2021-02-17T01:54:08Z","receivedAt":"2021-02-17T01:54:55Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff Hostetler writes:\n\n> On 2/3/21 10:34 AM, Jeff Hostetler via GitGitGadget wrote:\n>> Here is version 2 of this series.\n>\n> I didn't see this series in the \"what's cooking\" emails and\n> was wondering if there was something that I still needed to\n> attend to.\n\nI think it fell through the cracks. It seems that Taylor gave a\nreviewed-by for the whole set in v2 but as a reply to the cover\nletter of the original series, and no messages in v2 saw any\ncomments.\n\nI just picked them up and queued. Thanks for pinging.\n\n"}]}