git/list[1] front-page[2] threads[3] people[4] search[5] about
 

[PATCH] Add a new lstat implementation based on Win32 API, and make stat use that implementation too.

From
Marius Storm-Olsen <marius@trolltech.com>
Date
Sep 2, 2007, 14:51 UTC
Message-ID
<46DACE0D.5070501@trolltech.com>
In-Reply-To
<46DACD93.9000509@trolltech.com>

This gives us a significant speedup when adding, committing and stat'ing files. (Also, since Windows doesn't really handle symlinks, it's fine that stat just uses lstat)

Signed-off-by: Marius Storm-Olsen <mstormo_git@storm-olsen.com>
---
The following patch will override the normal Posix implementation of stat and
lstat on Windows, and use normal Windows API to ensure we're stat'ing as fast
as possible. With this patch I get the performance increase to the far right.

Initially, I only replaced lstat, since that's what the MinGW port of Git had implemented from before. But, since we don't really care about symlinks on Windows, I decided to simply use the same implementation for stat as well. The performance benefit is clearly indicated in the report below.

 With normal lstat & stat   lstat, based on Win32      lstat & stat, as Win32
 -------------------------  -------------------------  -------------------------
 Command: git init          Command: git init          Command: git init
 -------------------------  -------------------------  -------------------------
 real    0m0.047s           real      0m0.047s         real       0m0.078s
 user    0m0.031s           user      0m0.031s         user       0m0.031s
 sys     0m0.000s           sys       0m0.000s         sys        0m0.000s
 -------------------------  -------------------------  -------------------------
 Command: git add .         Command: git add .         Command: git add .
 -------------------------  -------------------------  -------------------------
 real    0m19.390s          real      0m19.390s        real       0m12.187s
 user    0m0.015s           user      0m0.015s         user       0m0.015s
 sys     0m0.030s           sys       0m0.030s         sys        0m0.015s
 -------------------------  -------------------------  -------------------------
 Command: git commit -a...  Command: git commit -a...  Command: git commit -a...
 -------------------------  -------------------------  -------------------------
 real    0m30.812s          real      0m22.547s        real       0m17.297s
 user    0m0.015s           user      0m0.031s         user       0m0.015s
 sys     0m0.000s           sys       0m0.000s         sys        0m0.015s
 -------------------------  -------------------------  -------------------------
 3x Command: git-status     3x Command: git-status     3x Command: git-status
 -------------------------  -------------------------  -------------------------
 real    0m11.860s          real      0m5.360s         real       0m5.344s
 user    0m0.015s           user      0m0.015s         user       0m0.015s
 sys     0m0.015s           sys       0m0.015s         sys        0m0.031s
 real    0m11.703s          real      0m5.312s         real       0m5.390s
 user    0m0.015s           user      0m0.015s         user       0m0.031s
 sys     0m0.000s           sys       0m0.000s         sys        0m0.000s
 real    0m11.672s          real      0m5.359s         real       0m5.344s
 user    0m0.031s           user      0m0.015s         user       0m0.015s
 sys     0m0.000s           sys       0m0.015s         sys        0m0.016s
 -------------------------  -------------------------  -------------------------
 Command: git commit...     Command: git commit...     Command: git commit...
 (single file)              (single file)              (single file)
 -------------------------  -------------------------  -------------------------
 real    0m14.234s          real      0m7.969s         real       0m7.875s
 user    0m0.015s           user      0m0.015s         user       0m0.015s
 sys     0m0.000s           sys       0m0.016s         sys        0m0.000s
 compat/mingw.c    |   52 ++++++++++++++++++++++++++++++++++++++++++++--------
 git-compat-util.h |    5 +++++
 2 files changed, 49 insertions(+), 8 deletions(-)
diff --git a/compat/mingw.c b/compat/mingw.c
index 7711a3f..207378c 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -23,19 +23,52 @@ int fchmod(int fildes, mode_t mode)
 	return -1;
 }

-int lstat(const char *file_name, struct stat *buf)
+static inline time_t filetime_to_time_t(const FILETIME *ft)
+{
+	long long winTime = ((long long)ft->dwHighDateTime << 32) + ft->dwLowDateTime;
+	winTime -= 116444736000000000LL; /* Windows to Unix Epoch conversion */
+	winTime /= 10000000;		 /* Nano to seconds resolution */
+	return (time_t)winTime;
+}
+
+extern int _getdrive( void );
+int git_lstat(const char *file_name, struct stat *buf)
 {
 	int namelen;
 	static char alt_name[PATH_MAX];
-
-	if (!stat(file_name, buf))
+	char* ext;
+	WIN32_FILE_ATTRIBUTE_DATA fdata;
+
+	if (GetFileAttributesExA(file_name, GetFileExInfoStandard, &fdata)) {
+		int fMode = S_IREAD;
+		if (fdata.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY)
+			fMode |= S_IFDIR;
+		else {
+			fMode |= S_IFREG;
+		ext = strrchr(file_name, '.');
+		if (ext && (!_stricmp(ext, ".exe") ||
+			    !_stricmp(ext, ".com") ||
+			    !_stricmp(ext, ".bat") ||
+			    !_stricmp(ext, ".cmd")))
+			fMode |= S_IEXEC;
+		}
+		if (!(fdata.dwFileAttributes & FILE_ATTRIBUTE_READONLY))
+			fMode |= S_IWRITE;
+
+		buf->st_ino = 0;
+		buf->st_gid = 0;
+		buf->st_uid = 0;
+		buf->st_nlink = 1;
+		buf->st_mode = fMode;
+		buf->st_size = fdata.nFileSizeLow; /* Can't use nFileSizeHigh, since it's not a stat64 */
+		buf->st_dev = buf->st_rdev = (_getdrive() - 1);
+		buf->st_atime = filetime_to_time_t(&(fdata.ftLastAccessTime));
+		buf->st_mtime = filetime_to_time_t(&(fdata.ftLastWriteTime));
+		buf->st_ctime = filetime_to_time_t(&(fdata.ftCreationTime));
 		return 0;
+	}
+	errno = ENOENT;

-	/* if file_name ended in a '/', Windows returned ENOENT;
-	 * try again without trailing slashes
-	 */
-	if (errno != ENOENT)
-		return -1;
 	namelen = strlen(file_name);
 	if (namelen && file_name[namelen-1] != '/')
 		return -1;
@@ -47,6 +80,9 @@ int lstat(const char *file_name, struct stat *buf)
 	alt_name[namelen] = 0;
 	return stat(alt_name, buf);
 }
+int git_stat(const char *file_name, struct stat *buf) {
+    return git_lstat(file_name, buf);
+}

 /* missing: link, mkstemp, fchmod, getuid (?), gettimeofday */
 int socketpair(int d, int type, int protocol, int sv[2])
diff --git a/git-compat-util.h b/git-compat-util.h
index 1ba499f..de1f062 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -488,6 +488,11 @@ int mingw_rename(const char*, const char*);
 extern void quote_argv(const char **dst, const char **src);
 extern const char *parse_interpreter(const char *cmd);

+/* Make git on Windows use git_lstat and git_stat instead of lstat and stat */
+int git_lstat(const char *file_name, struct stat *buf);
+int git_stat(const char *file_name, struct stat *buf);
+#define lstat(x,y) git_lstat(x,y)
+#define stat(x,y) git_stat(x,y)
 #endif /* __MINGW32__ */

 #endif
--
mingw.v1.5.2.4.1311.g376df-dirty
Previous: Marius Storm-OlsenNext: Marius Storm-Olsen
Message 2 of 86 in “Stats in Git”
  1. Marius Storm-OlsenSep 2, 2007
  2. Add a new lstat implementation based on Win32 API, and make stat use that implementation too.Marius Storm-Olsen, Sep 2, 2007
  3. Marius Storm-OlsenSep 2, 2007
  4. Reece DunnSep 2, 2007
  5. Marius Storm-OlsenSep 2, 2007
  6. Reece DunnSep 2, 2007
  7. Brian GernhardtSep 2, 2007
  8. Reece DunnSep 2, 2007
  9. Marius Storm-OlsenSep 2, 2007
  10. Johannes SchindelinSep 2, 2007
  11. David KastrupSep 2, 2007
  12. Marius Storm-OlsenSep 2, 2007
  13. Johannes SixtSep 2, 2007
  14. Marius Storm-OlsenSep 2, 2007
  15. Johannes SixtSep 2, 2007
  16. Add a new lstat implementation based on Win32 API, and make stat use that implementation too.Marius Storm-Olsen, Sep 2, 2007
  17. Robin RosenbergSep 2, 2007
  18. Johannes SchindelinSep 2, 2007
  19. Robin RosenbergSep 2, 2007
  20. Johannes SchindelinSep 2, 2007
  21. Johannes SixtSep 3, 2007
  22. Miklos VajnaSep 3, 2007
  23. David KastrupSep 3, 2007
  24. Miklos VajnaSep 5, 2007
  25. David KastrupSep 5, 2007
  26. Miklos VajnaSep 6, 2007
  27. David KastrupSep 6, 2007
  28. Douglas StockwellSep 6, 2007
  29. David KastrupSep 7, 2007
  30. Alex RiesenSep 2, 2007
  31. Robin RosenbergSep 2, 2007
  32. Marius Storm-OlsenSep 3, 2007
  33. Johannes SchindelinSep 3, 2007
  34. David KastrupSep 3, 2007
  35. Marius Storm-OlsenSep 3, 2007
  36. Johannes SchindelinSep 3, 2007
  37. Alex RiesenSep 2, 2007
  38. Marius Storm-OlsenSep 3, 2007
  39. Johannes SixtSep 3, 2007
  40. Marius Storm-OlsenSep 3, 2007
  41. Alex RiesenSep 2, 2007
  42. Marius Storm-OlsenSep 2, 2007
  43. Matthieu MoySep 3, 2007
  44. Marius Storm-OlsenSep 3, 2007
  45. Johannes SchindelinSep 3, 2007
  46. Marius Storm-OlsenSep 3, 2007
  47. Johannes SchindelinSep 3, 2007
  48. Marius Storm-OlsenSep 3, 2007
  49. Johannes SchindelinSep 3, 2007
  50. Johannes SixtSep 3, 2007
  51. Johannes SchindelinSep 3, 2007
  52. Marius Storm-OlsenSep 3, 2007
  53. Johannes SchindelinSep 4, 2007
  54. Johannes SixtSep 4, 2007
  55. David KastrupSep 4, 2007
  56. Marius Storm-OlsenSep 4, 2007
  57. Johannes SixtSep 4, 2007
  58. Marius Storm-OlsenSep 4, 2007
  59. Johannes SixtSep 4, 2007
  60. David KastrupSep 4, 2007
  61. Johannes SchindelinSep 4, 2007
  62. Johannes SixtSep 4, 2007
  63. Marius Storm-OlsenSep 4, 2007
  64. Marius Storm-OlsenSep 4, 2007
  65. Johannes SixtSep 4, 2007
  66. Johannes SchindelinSep 4, 2007
  67. Johannes SixtSep 4, 2007
  68. Johannes SchindelinSep 4, 2007
  69. Johannes SchindelinSep 4, 2007
  70. Marius Storm-OlsenSep 4, 2007
  71. Johannes SchindelinSep 4, 2007
  72. David KastrupSep 4, 2007
  73. Marius Storm-OlsenSep 4, 2007
  74. Johannes SchindelinSep 4, 2007
  75. Johannes SchindelinSep 4, 2007
  76. Rutger NijlunsingSep 4, 2007
  77. Reece DunnSep 4, 2007
  78. Marius Storm-OlsenSep 5, 2007
  79. Johannes SchindelinSep 5, 2007
  80. Johannes SixtSep 4, 2007
  81. Johannes SixtSep 6, 2007
  82. Marius Storm-OlsenSep 6, 2007
  83. Johannes SixtSep 3, 2007
  84. Johannes SchindelinSep 3, 2007
  85. Marius Storm-OlsenSep 3, 2007
  86. Johannes SchindelinSep 3, 2007

Read the whole thread, see it on lore, or plain text.

$ cat FOOTERMessages come from the public archive at lore.kernel.org/git, fetched every hour. The front page is chosen and written each morning by an AI editor and can be wrong; the threads themselves are the record. About and API. For agents: an MCP server at https://gitlist.dev/mcp, and any thread, story or person page as Markdown by adding .md to its URL (or sending Accept: text/markdown). Details in /llms.txt.