git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH v2 14/16] fsmonitor: support case-insensitive events

From
JGJeff Hostetler via GitGitGadget <gitgitgadget@gmail.com>
Date
Feb 23, 2024, 03:18 UTC
Message-ID
<288f3f4e54e98a68d72e97125b1520605c138c3c.1708658300.git.gitgitgadget@gmail.com>
In-Reply-To
<pull.1662.v2.git.1708658300.gitgitgadget@gmail.com>
From: Jeff Hostetler <jeffhostetler@github.com>

Teach fsmonitor_refresh_callback() to handle case-insensitive lookups if case-sensitive lookups fail on case-insensitive systems. This can cause 'git status' to report stale status for files if there are case issues/errors in the worktree.

The FSMonitor daemon sends FSEvents using the observed spelling of each pathname. On case-insensitive file systems this may be different than the expected case spelling.

The existing code uses index_name_pos() to find the cache-entry for the pathname in the FSEvent and clear the CE_FSMONITOR_VALID bit so that the worktree scan/index refresh will revisit and revalidate the path.

On a case-insensitive file system, the exact match lookup may fail to find the associated cache-entry. This causes status to think that the cached CE flags are correct and skip over the file.

Update event handling to optionally use the name-hash and dir-name-hash if necessary.

Signed-off-by: Jeff Hostetler <jeffhostetler@github.com>
---
 fsmonitor.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)
diff --git a/fsmonitor.c b/fsmonitor.c
index 739ddbf7aca..ac638a61c00 100644
--- a/fsmonitor.c
+++ b/fsmonitor.c
@@ -5,6 +5,7 @@
 #include "ewah/ewok.h"
 #include "fsmonitor.h"
 #include "fsmonitor-ipc.h"
+#include "name-hash.h"
 #include "run-command.h"
 #include "strbuf.h"
 #include "trace2.h"
@@ -186,6 +187,102 @@ static int query_fsmonitor_hook(struct repository *r,
 static size_t handle_path_with_trailing_slash(
 	struct index_state *istate, const char *name, int pos);
 
+/*
+ * Use the name-hash to do a case-insensitive cache-entry lookup with
+ * the pathname and invalidate the cache-entry.
+ *
+ * Returns the number of cache-entries that we invalidated.
+ */
+static size_t handle_using_name_hash_icase(
+	struct index_state *istate, const char *name)
+{
+	struct cache_entry *ce = NULL;
+
+	ce = index_file_exists(istate, name, strlen(name), 1);
+	if (!ce)
+		return 0;
+
+	/*
+	 * A case-insensitive search in the name-hash using the
+	 * observed pathname found a cache-entry, so the observed path
+	 * is case-incorrect.  Invalidate the cache-entry and use the
+	 * correct spelling from the cache-entry to invalidate the
+	 * untracked-cache.  Since we now have sparse-directories in
+	 * the index, the observed pathname may represent a regular
+	 * file or a sparse-index directory.
+	 *
+	 * Note that we should not have seen FSEvents for a
+	 * sparse-index directory, but we handle it just in case.
+	 *
+	 * Either way, we know that there are not any cache-entries for
+	 * children inside the cone of the directory, so we don't need to
+	 * do the usual scan.
+	 */
+	trace_printf_key(&trace_fsmonitor,
+			 "fsmonitor_refresh_callback MAP: '%s' '%s'",
+			 name, ce->name);
+
+	untracked_cache_invalidate_trimmed_path(istate, ce->name, 0);
+
+	ce->ce_flags &= ~CE_FSMONITOR_VALID;
+	return 1;
+}
+
+/*
+ * Use the dir-name-hash to find the correct-case spelling of the
+ * directory.  Use the canonical spelling to invalidate all of the
+ * cache-entries within the matching cone.
+ *
+ * Returns the number of cache-entries that we invalidated.
+ */
+static size_t handle_using_dir_name_hash_icase(
+	struct index_state *istate, const char *name)
+{
+	struct strbuf canonical_path = STRBUF_INIT;
+	int pos;
+	size_t len = strlen(name);
+	size_t nr_in_cone;
+
+	if (name[len - 1] == '/')
+		len--;
+
+	if (!index_dir_find(istate, name, len, &canonical_path))
+		return 0; /* name is untracked */
+
+	if (!memcmp(name, canonical_path.buf, canonical_path.len)) {
+		strbuf_release(&canonical_path);
+		/*
+		 * NEEDSWORK: Our caller already tried an exact match
+		 * and failed to find one.  They called us to do an
+		 * ICASE match, so we should never get an exact match,
+		 * so we could promote this to a BUG() here if we
+		 * wanted to.  It doesn't hurt anything to just return
+		 * 0 and go on becaus we should never get here.  Or we
+		 * could just get rid of the memcmp() and this "if"
+		 * clause completely.
+		 */
+		return 0; /* should not happen */
+	}
+
+	trace_printf_key(&trace_fsmonitor,
+			 "fsmonitor_refresh_callback MAP: '%s' '%s'",
+			 name, canonical_path.buf);
+
+	/*
+	 * The dir-name-hash only tells us the corrected spelling of
+	 * the prefix.  We have to use this canonical path to do a
+	 * lookup in the cache-entry array so that we repeat the
+	 * original search using the case-corrected spelling.
+	 */
+	strbuf_addch(&canonical_path, '/');
+	pos = index_name_pos(istate, canonical_path.buf,
+			     canonical_path.len);
+	nr_in_cone = handle_path_with_trailing_slash(
+		istate, canonical_path.buf, pos);
+	strbuf_release(&canonical_path);
+	return nr_in_cone;
+}
+
 /*
  * The daemon sent an observed pathname without a trailing slash.
  * (This is the normal case.)  We do not know if it is a tracked or
@@ -319,6 +416,19 @@ static void fsmonitor_refresh_callback(struct index_state *istate, char *name)
 	else
 		nr_in_cone = handle_path_without_trailing_slash(istate, name, pos);
 
+	/*
+	 * If we did not find an exact match for this pathname or any
+	 * cache-entries with this directory prefix and we're on a
+	 * case-insensitive file system, try again using the name-hash
+	 * and dir-name-hash.
+	 */
+	if (!nr_in_cone && ignore_case) {
+		nr_in_cone = handle_using_name_hash_icase(istate, name);
+		if (!nr_in_cone)
+			nr_in_cone = handle_using_dir_name_hash_icase(
+				istate, name);
+	}
+
 	if (nr_in_cone)
 		trace_printf_key(&trace_fsmonitor,
 				 "fsmonitor_refresh_callback CNT: %d",
-- 
gitgitgadget
Previous: Junio C HamanoNext: Junio C Hamano
Message 62 of 91 in “FSMonitor edge cases on case-insensitive file systems”
  1. 00/12 FSMonitor edge cases on case-insensitive file systemsJeff Hostetler via GitGitGadget, Feb 13, 2024
  2. 01/12 sparse-index: pass string length to index_file_exists()Jeff Hostetler via GitGitGadget, Feb 13, 2024
  3. Junio C HamanoFeb 13, 2024
  4. Jeff HostetlerFeb 20, 2024
  5. 02/12 name-hash: add index_dir_exists2()Jeff Hostetler via GitGitGadget, Feb 13, 2024
  6. Junio C HamanoFeb 13, 2024
  7. Jeff HostetlerFeb 20, 2024
  8. Junio C HamanoFeb 20, 2024
  9. Patrick SteinhardtFeb 15, 2024
  10. 03/12 t7527: add case-insensitve test for FSMonitorJeff Hostetler via GitGitGadget, Feb 13, 2024
  11. 04/12 fsmonitor: refactor refresh callback on directory eventsJeff Hostetler via GitGitGadget, Feb 13, 2024
  12. Patrick SteinhardtFeb 15, 2024
  13. Jeff HostetlerFeb 20, 2024
  14. Patrick SteinhardtFeb 21, 2024
  15. 05/12 fsmonitor: refactor refresh callback for non-directory eventsJeff Hostetler via GitGitGadget, Feb 13, 2024
  16. Junio C HamanoFeb 14, 2024
  17. Patrick SteinhardtFeb 15, 2024
  18. 06/12 fsmonitor: clarify handling of directory events in callbackJeff Hostetler via GitGitGadget, Feb 13, 2024
  19. Junio C HamanoFeb 14, 2024
  20. Jeff HostetlerFeb 20, 2024
  21. Junio C HamanoFeb 20, 2024
  22. Patrick SteinhardtFeb 15, 2024
  23. Jeff HostetlerFeb 20, 2024
  24. 07/12 fsmonitor: refactor untracked-cache invalidationJeff Hostetler via GitGitGadget, Feb 13, 2024
  25. Junio C HamanoFeb 14, 2024
  26. Patrick SteinhardtFeb 15, 2024
  27. 08/12 fsmonitor: support case-insensitive directory eventsJeff Hostetler via GitGitGadget, Feb 13, 2024
  28. Patrick SteinhardtFeb 15, 2024
  29. 09/12 fsmonitor: refactor non-directory callbackJeff Hostetler via GitGitGadget, Feb 13, 2024
  30. Patrick SteinhardtFeb 15, 2024
  31. 10/12 fsmonitor: support case-insensitive non-directory eventsJeff Hostetler via GitGitGadget, Feb 13, 2024
  32. 11/12 fsmonitor: refactor bit invalidation in refresh callbackJeff Hostetler via GitGitGadget, Feb 13, 2024
  33. Patrick SteinhardtFeb 15, 2024
  34. 12/12 t7527: update case-insenstive fsmonitor testJeff Hostetler via GitGitGadget, Feb 13, 2024
  35. 00/16 FSMonitor edge cases on case-insensitive file systemsJeff Hostetler via GitGitGadget, Feb 23, 2024
  36. 01/16 name-hash: add index_dir_find()Jeff Hostetler via GitGitGadget, Feb 23, 2024
  37. Junio C HamanoFeb 23, 2024
  38. 03/16 t7527: temporarily disable case-insensitive testsJeff Hostetler via GitGitGadget, Feb 23, 2024
  39. Junio C HamanoFeb 23, 2024
  40. Jeff HostetlerFeb 26, 2024
  41. 02/16 t7527: add case-insensitve test for FSMonitorJeff Hostetler via GitGitGadget, Feb 23, 2024
  42. 05/16 fsmonitor: clarify handling of directory events in callback helperJeff Hostetler via GitGitGadget, Feb 23, 2024
  43. 04/16 fsmonitor: refactor refresh callback on directory eventsJeff Hostetler via GitGitGadget, Feb 23, 2024
  44. Junio C HamanoFeb 23, 2024
  45. 06/16 fsmonitor: refactor refresh callback for non-directory eventsJeff Hostetler via GitGitGadget, Feb 23, 2024
  46. Junio C HamanoFeb 23, 2024
  47. Torsten BögershausenFeb 25, 2024
  48. Junio C HamanoFeb 25, 2024
  49. 07/16 dir: create untracked_cache_invalidate_trimmed_path()Jeff Hostetler via GitGitGadget, Feb 23, 2024
  50. Torsten BögershausenFeb 25, 2024
  51. 08/16 fsmonitor: refactor untracked-cache invalidationJeff Hostetler via GitGitGadget, Feb 23, 2024
  52. 09/16 fsmonitor: move untracked invalidation into helper functionsJeff Hostetler via GitGitGadget, Feb 23, 2024
  53. Junio C HamanoFeb 23, 2024
  54. Jeff HostetlerFeb 26, 2024
  55. 10/16 fsmonitor: return invalidated cache-entry count on directory eventJeff Hostetler via GitGitGadget, Feb 23, 2024
  56. 11/16 fsmonitor: remove custom loop from non-directory path handlerJeff Hostetler via GitGitGadget, Feb 23, 2024
  57. Junio C HamanoFeb 23, 2024
  58. 13/16 fsmonitor: trace the new invalidated cache-entry countJeff Hostetler via GitGitGadget, Feb 23, 2024
  59. Junio C HamanoFeb 23, 2024
  60. 12/16 fsmonitor: return invalided cache-entry count on non-directory eventJeff Hostetler via GitGitGadget, Feb 23, 2024
  61. Junio C HamanoFeb 23, 2024
  62. 14/16 fsmonitor: support case-insensitive eventsJeff Hostetler via GitGitGadget, Feb 23, 2024
  63. Junio C HamanoFeb 23, 2024
  64. Jeff HostetlerFeb 26, 2024
  65. Junio C HamanoFeb 26, 2024
  66. Torsten BögershausenFeb 25, 2024
  67. Jeff HostetlerFeb 26, 2024
  68. 15/16 fsmonitor: refactor bit invalidation in refresh callbackJeff Hostetler via GitGitGadget, Feb 23, 2024
  69. Junio C HamanoFeb 23, 2024
  70. 16/16 t7527: update case-insenstive fsmonitor testJeff Hostetler via GitGitGadget, Feb 23, 2024
  71. 00/14 FSMonitor edge cases on case-insensitive file systemsJeff Hostetler via GitGitGadget, Feb 26, 2024
  72. 01/14 name-hash: add index_dir_find()Jeff Hostetler via GitGitGadget, Feb 26, 2024
  73. 02/14 t7527: add case-insensitve test for FSMonitorJeff Hostetler via GitGitGadget, Feb 26, 2024
  74. 03/14 fsmonitor: refactor refresh callback on directory eventsJeff Hostetler via GitGitGadget, Feb 26, 2024
  75. 04/14 fsmonitor: clarify handling of directory events in callback helperJeff Hostetler via GitGitGadget, Feb 26, 2024
  76. 05/14 fsmonitor: refactor refresh callback for non-directory eventsJeff Hostetler via GitGitGadget, Feb 26, 2024
  77. 06/14 dir: create untracked_cache_invalidate_trimmed_path()Jeff Hostetler via GitGitGadget, Feb 26, 2024
  78. 07/14 fsmonitor: refactor untracked-cache invalidationJeff Hostetler via GitGitGadget, Feb 26, 2024
  79. 08/14 fsmonitor: move untracked-cache invalidation into helper functionsJeff Hostetler via GitGitGadget, Feb 26, 2024
  80. 09/14 fsmonitor: return invalidated cache-entry count on directory eventJeff Hostetler via GitGitGadget, Feb 26, 2024
  81. 10/14 fsmonitor: remove custom loop from non-directory path handlerJeff Hostetler via GitGitGadget, Feb 26, 2024
  82. 11/14 fsmonitor: return invalided cache-entry count on non-directory eventJeff Hostetler via GitGitGadget, Feb 26, 2024
  83. Patrick SteinhardtMar 6, 2024
  84. 12/14 fsmonitor: trace the new invalidated cache-entry countJeff Hostetler via GitGitGadget, Feb 26, 2024
  85. 13/14 fsmonitor: refactor bit invalidation in refresh callbackJeff Hostetler via GitGitGadget, Feb 26, 2024
  86. 14/14 fsmonitor: support case-insensitive eventsJeff Hostetler via GitGitGadget, Feb 26, 2024
  87. Patrick SteinhardtMar 6, 2024
  88. Junio C HamanoFeb 27, 2024
  89. Patrick SteinhardtMar 6, 2024
  90. Junio C HamanoMar 6, 2024
  91. Jeff HostetlerMar 6, 2024

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.