git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH 01/02/RFC] implement a stat cache

From
LRLuciano Rocha <luciano@eurotux.com>
Date
Apr 20, 2008, 11:13 UTC
Message-ID
<20080420111346.GA13411@bit.office.eurotux.com>
In-Reply-To
<alpine.LFD.1.10.0804191515120.2779@woody.linux-foundation.org>

An implementation of stat(2) and lstat(2) caching. Both the return code and returned information are cached.

Signed-off-by: Luciano Rocha <strange@nsk.no-ip.org>
---
On Sat, Apr 19, 2008 at 03:39:37PM -0700, Linus Torvalds wrote:
> Yeah. I didn't look any further, but we do a total of *nine* 'lstat()' 
> calls for each file we know about that is dirty, and *seven* when they are 
> clean. Plus maybe a few more.
That's a lot. Why not use a stat cache?

With these changes, my git status . in WebKit changes from 28.215s to 15.414s.

 stat-cache.c |   69 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 stat-cache.h |    9 +++++++
 2 files changed, 78 insertions(+), 0 deletions(-)
 create mode 100644 stat-cache.c
 create mode 100644 stat-cache.h
diff --git a/stat-cache.c b/stat-cache.c
new file mode 100644
index 0000000..6a33cec
--- /dev/null
+++ b/stat-cache.c
@@ -0,0 +1,69 @@
+/*
+ * Cache (l)stat operations
+ */
+
+#include "stat-cache.h"
+#include "hash.h"
+#include "path-list.h";
+
+static struct hash_table stat_cache;
+static struct hash_table lstat_cache;
+
+struct stat_result {
+	struct stat st;
+	int ret;
+};
+
+/* based on hash_name from read_cache.c */
+static unsigned int hash_path(const char *path)
+{
+	unsigned int hash = 0x123;
+
+	while (*path)
+		hash = hash*101 + *path++;
+	return hash;
+}
+
+/* cache is HASH->PATH-LIST->(return code, struct stat) */
+static int cached_stat(int (*f)(const char *, struct stat *),
+		struct hash_table *ht, const char *path, struct stat *buf)
+{
+	unsigned int hash;
+	struct path_list *list;
+	struct path_list_item *cached;
+	struct stat_result *result;
+
+	hash = hash_path(path);
+
+	list = lookup_hash(hash, ht);
+
+	if (!list) {
+		list = xcalloc(1, sizeof *list);
+		list->strdup_paths = 1;
+		insert_hash(hash, list, ht);
+	}
+
+	cached = path_list_lookup(path, list);
+
+	if (cached) {
+		result = cached->util;
+	} else {
+		result = xmalloc(sizeof *result);
+		result->ret = f(path, &result->st);
+		path_list_insert(path, list)->util = result;
+	}
+
+	if (result->ret == 0)
+		memcpy(buf, &result->st, sizeof *buf);
+	return result->ret;
+}
+
+int cstat(const char *path, struct stat *buf)
+{
+	return cached_stat(stat, &stat_cache, path, buf);
+}
+
+int clstat(const char *path, struct stat *buf)
+{
+	return cached_stat(lstat, &lstat_cache, path, buf);
+}
diff --git a/stat-cache.h b/stat-cache.h
new file mode 100644
index 0000000..754348f
--- /dev/null
+++ b/stat-cache.h
@@ -0,0 +1,9 @@
+#ifndef STAT_CACHE_H
+#define STAT_CACHE_H
+
+#include "git-compat-util.h"
+
+int cstat(const char *path, struct stat *buf);
+int clstat(const char *path, struct stat *buf);
+
+#endif /* STAT_CACHE_H */
-- 
1.5.5.76.gbb45.dirty
Previous: Junio C HamanoNext: Luciano Rocha
Message 10 of 39 in “Git performance on OS X”
  1. Pieter de BieApr 19, 2008
  2. Linus TorvaldsApr 19, 2008
  3. Linus TorvaldsApr 19, 2008
  4. Pieter de BieApr 19, 2008
  5. David KastrupApr 20, 2008
  6. Linus TorvaldsApr 19, 2008
  7. Pieter de BieApr 19, 2008
  8. Linus TorvaldsApr 19, 2008
  9. Junio C HamanoApr 20, 2008
  10. 01/02 implement a stat cacheLuciano Rocha, Apr 20, 2008
  11. 02/02 make use of the stat cacheLuciano Rocha, Apr 20, 2008
  12. Luciano RochaApr 20, 2008
  13. Linus TorvaldsApr 20, 2008
  14. Luciano RochaApr 20, 2008
  15. Linus TorvaldsApr 20, 2008
  16. Linus TorvaldsApr 20, 2008
  17. Dmitry PotapovApr 21, 2008
  18. Johan HerlandApr 21, 2008
  19. Junio C HamanoApr 21, 2008
  20. Linus TorvaldsApr 21, 2008
  21. Linus TorvaldsApr 21, 2008
  22. Junio C HamanoApr 21, 2008
  23. Linus TorvaldsApr 21, 2008
  24. Junio C HamanoApr 21, 2008
  25. David KastrupApr 21, 2008
  26. Jakub NarebskiApr 19, 2008
  27. Linus TorvaldsApr 19, 2008
  28. Linus TorvaldsApr 19, 2008
  29. Pieter de BieApr 19, 2008
  30. Linus TorvaldsApr 19, 2008
  31. Roman ShaposhnikApr 19, 2008
  32. Pieter de BieApr 19, 2008
  33. Linus TorvaldsApr 20, 2008
  34. Roman ShaposhnikApr 20, 2008
  35. Pieter de BieApr 19, 2008
  36. Linus TorvaldsApr 20, 2008
  37. Dmitry PotapovApr 20, 2008
  38. David KastrupApr 20, 2008
  39. Linus TorvaldsApr 19, 2008

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.