{"thread":{"id":"38553","subject":"[PATCH 0/3] Win32: nanosecond-precision file times","startedAt":"2015-02-11T23:49:57Z","lastAt":"2015-02-17T21:57:45Z","messageCount":15,"participants":["Karsten Blees","Junio C Hamano","Johannes Schindelin","Thomas Braun"],"isPatch":true,"patchVersion":1,"patchTotal":3},"messages":[{"id":"255954","messageId":"54DBEAA5.6000205@gmail.com","threadId":"38553","inReplyTo":null,"subject":"[PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-11T23:49:57Z","receivedAt":"2015-02-11T23:49:57Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"This patch series was inspired by the problem that Git does not\ndetect changed file content if st_size, st_mtime and st_ctime\nare unchanged. This was apparently caused by VSS2Git resetting\nmtime to a value in the past. [1]\n\nI believe (or rather hope) that all involved in the discussion\nagree that Git cannot reasonably be expected to detect changed\nfile content if file time(s) are reset on purpose.\n\nHowever, some users have expressed concerns that 'same size and\nmtime' [2] may theoretically happen by chance in daily operation.\n\nThis patch series adopts POSIX 2013 'struct timespec' file times\nto make this practically impossible, at least on NTFS with 100ns\nfile time resolution.\n\nCheers,\nKarsten\n\n[1] https://github.com/msysgit/git/issues/312\n[2] Note that st_ctime of a file never changes on Windows, as it\n    means 'creation time' rather than 'change status time'.\n\nKarsten Blees (3):\n  Win32: make FILETIME conversion functions public\n  Win32: replace MSVCRT's fstat() with a Win32-based implementation\n  Win32: implement nanosecond-precision file times\n\n compat/mingw.c   | 56 +++++++++++++++++++++++++-------------------------------\n compat/mingw.h   | 55 +++++++++++++++++++++++++++++++++++++++++++++----------\n config.mak.uname |  4 ++--\n 3 files changed, 72 insertions(+), 43 deletions(-)\n\n-- \n2.3.0.3.ge7778af\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"255955","messageId":"54DBEAED.5030201@gmail.com","threadId":"38553","inReplyTo":"54DBEAA5.6000205@gmail.com","subject":"[PATCH 1/3] Win32: make FILETIME conversion functions public","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-11T23:51:09Z","receivedAt":"2015-02-11T23:51:09Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"Signed-off-by: Karsten Blees <blees@dcon.de>\n---\n compat/mingw.c | 16 ----------------\n compat/mingw.h | 16 ++++++++++++++++\n 2 files changed, 16 insertions(+), 16 deletions(-)\n\ndiff --git a/compat/mingw.c b/compat/mingw.c\nindex 70f3191..ba3cfb0 100644\n--- a/compat/mingw.c\n+++ b/compat/mingw.c\n@@ -419,22 +419,6 @@ int mingw_chmod(const char *filename, int mode)\n \treturn _wchmod(wfilename, mode);\n }\n \n-/*\n- * The unit of FILETIME is 100-nanoseconds since January 1, 1601, UTC.\n- * Returns the 100-nanoseconds (\"hekto nanoseconds\") since the epoch.\n- */\n-static inline long long filetime_to_hnsec(const FILETIME *ft)\n-{\n-\tlong long winTime = ((long long)ft->dwHighDateTime << 32) + ft->dwLowDateTime;\n-\t/* Windows to Unix Epoch conversion */\n-\treturn winTime - 116444736000000000LL;\n-}\n-\n-static inline time_t filetime_to_time_t(const FILETIME *ft)\n-{\n-\treturn (time_t)(filetime_to_hnsec(ft) / 10000000);\n-}\n-\n /* We keep the do_lstat code in a separate function to avoid recursion.\n  * When a path ends with a slash, the stat will fail with ENOENT. In\n  * this case, we strip the trailing slashes and stat again.\ndiff --git a/compat/mingw.h b/compat/mingw.h\nindex 5e499cf..f2a78b4 100644\n--- a/compat/mingw.h\n+++ b/compat/mingw.h\n@@ -283,6 +283,22 @@ static inline int getrlimit(int resource, struct rlimit *rlp)\n }\n \n /*\n+ * The unit of FILETIME is 100-nanoseconds since January 1, 1601, UTC.\n+ * Returns the 100-nanoseconds (\"hekto nanoseconds\") since the epoch.\n+ */\n+static inline long long filetime_to_hnsec(const FILETIME *ft)\n+{\n+\tlong long winTime = ((long long)ft->dwHighDateTime << 32) + ft->dwLowDateTime;\n+\t/* Windows to Unix Epoch conversion */\n+\treturn winTime - 116444736000000000LL;\n+}\n+\n+static inline time_t filetime_to_time_t(const FILETIME *ft)\n+{\n+\treturn (time_t)(filetime_to_hnsec(ft) / 10000000);\n+}\n+\n+/*\n  * Use mingw specific stat()/lstat()/fstat() implementations on Windows.\n  */\n #define off_t off64_t\n-- \n2.3.0.3.ge7778af\n\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"255956","messageId":"54DBEB26.6020403@gmail.com","threadId":"38553","inReplyTo":"54DBEAA5.6000205@gmail.com","subject":"[PATCH 2/3] Win32: replace MSVCRT's fstat() with a Win32-based implementation","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-11T23:52:06Z","receivedAt":"2015-02-11T23:52:06Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"fstat() is the only stat-related CRT function for which we don't have a\nfull replacement yet (and thus the only reason to stick with MSVCRT's\n'struct stat' definition).\n\nFully implement fstat(), in preparation of implementing a POSIX 2013\ncompatible 'struct stat' with nanosecond-precision file times.\n\nSigned-off-by: Karsten Blees <blees@dcon.de>\n---\n compat/mingw.c | 28 +++++++++++++++++++---------\n 1 file changed, 19 insertions(+), 9 deletions(-)\n\ndiff --git a/compat/mingw.c b/compat/mingw.c\nindex ba3cfb0..6d73a3d 100644\n--- a/compat/mingw.c\n+++ b/compat/mingw.c\n@@ -532,28 +532,38 @@ int mingw_fstat(int fd, struct stat *buf)\n {\n \tHANDLE fh = (HANDLE)_get_osfhandle(fd);\n \tBY_HANDLE_FILE_INFORMATION fdata;\n+\tDWORD avail;\n \n \tif (fh == INVALID_HANDLE_VALUE) {\n \t\terrno = EBADF;\n \t\treturn -1;\n \t}\n-\t/* direct non-file handles to MS's fstat() */\n-\tif (GetFileType(fh) != FILE_TYPE_DISK)\n-\t\treturn _fstati64(fd, buf);\n \n-\tif (GetFileInformationByHandle(fh, &fdata)) {\n-\t\tbuf->st_ino = 0;\n-\t\tbuf->st_gid = 0;\n-\t\tbuf->st_uid = 0;\n-\t\tbuf->st_nlink = 1;\n+\t/* initialize stat fields */\n+\tmemset(buf, 0, sizeof(*buf));\n+\tbuf->st_nlink = 1;\n+\n+\tswitch (GetFileType(fh) & ~FILE_TYPE_REMOTE) {\n+\tcase FILE_TYPE_DISK:\n+\t\tif (!GetFileInformationByHandle(fh, &fdata))\n+\t\t\tbreak;\n \t\tbuf->st_mode = file_attr_to_st_mode(fdata.dwFileAttributes);\n \t\tbuf->st_size = fdata.nFileSizeLow |\n \t\t\t(((off_t)fdata.nFileSizeHigh)<<32);\n-\t\tbuf->st_dev = buf->st_rdev = 0; /* not used by Git */\n \t\tbuf->st_atime = filetime_to_time_t(&(fdata.ftLastAccessTime));\n \t\tbuf->st_mtime = filetime_to_time_t(&(fdata.ftLastWriteTime));\n \t\tbuf->st_ctime = filetime_to_time_t(&(fdata.ftCreationTime));\n \t\treturn 0;\n+\n+\tcase FILE_TYPE_CHAR:\n+\t\tbuf->st_mode = _S_IFCHR;\n+\t\treturn 0;\n+\n+\tcase FILE_TYPE_PIPE:\n+\t\tbuf->st_mode = _S_IFIFO;\n+\t\tif (PeekNamedPipe(fh, NULL, 0, NULL, &avail, NULL))\n+\t\t\tbuf->st_size = avail;\n+\t\treturn 0;\n \t}\n \terrno = EBADF;\n \treturn -1;\n-- \n2.3.0.3.ge7778af\n\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"255957","messageId":"54DBEB5C.5070804@gmail.com","threadId":"38553","inReplyTo":"54DBEAA5.6000205@gmail.com","subject":"[PATCH 3/3] Win32: implement nanosecond-precision file times","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-11T23:53:00Z","receivedAt":"2015-02-11T23:53:00Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"We no longer use any of MSVCRT's stat-functions, so there's no need to\nstick to a CRT-compatible 'struct stat' either.\n\nDefine and use our own POSIX-2013-compatible 'struct stat' with nanosecond-\nprecision file times.\n\nSigned-off-by: Karsten Blees <blees@dcon.de>\n---\n compat/mingw.c   | 12 ++++++------\n compat/mingw.h   | 43 +++++++++++++++++++++++++++++++------------\n config.mak.uname |  4 ++--\n 3 files changed, 39 insertions(+), 20 deletions(-)\n\ndiff --git a/compat/mingw.c b/compat/mingw.c\nindex 6d73a3d..e4d5e3f 100644\n--- a/compat/mingw.c\n+++ b/compat/mingw.c\n@@ -442,9 +442,9 @@ static int do_lstat(int follow, const char *file_name, struct stat *buf)\n \t\tbuf->st_size = fdata.nFileSizeLow |\n \t\t\t(((off_t)fdata.nFileSizeHigh)<<32);\n \t\tbuf->st_dev = buf->st_rdev = 0; /* not used by Git */\n-\t\tbuf->st_atime = filetime_to_time_t(&(fdata.ftLastAccessTime));\n-\t\tbuf->st_mtime = filetime_to_time_t(&(fdata.ftLastWriteTime));\n-\t\tbuf->st_ctime = filetime_to_time_t(&(fdata.ftCreationTime));\n+\t\tfiletime_to_timespec(&(fdata.ftLastAccessTime), &(buf->st_atim));\n+\t\tfiletime_to_timespec(&(fdata.ftLastWriteTime), &(buf->st_mtim));\n+\t\tfiletime_to_timespec(&(fdata.ftCreationTime), &(buf->st_ctim));\n \t\tif (fdata.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) {\n \t\t\tWIN32_FIND_DATAW findbuf;\n \t\t\tHANDLE handle = FindFirstFileW(wfilename, &findbuf);\n@@ -550,9 +550,9 @@ int mingw_fstat(int fd, struct stat *buf)\n \t\tbuf->st_mode = file_attr_to_st_mode(fdata.dwFileAttributes);\n \t\tbuf->st_size = fdata.nFileSizeLow |\n \t\t\t(((off_t)fdata.nFileSizeHigh)<<32);\n-\t\tbuf->st_atime = filetime_to_time_t(&(fdata.ftLastAccessTime));\n-\t\tbuf->st_mtime = filetime_to_time_t(&(fdata.ftLastWriteTime));\n-\t\tbuf->st_ctime = filetime_to_time_t(&(fdata.ftCreationTime));\n+\t\tfiletime_to_timespec(&(fdata.ftLastAccessTime), &(buf->st_atim));\n+\t\tfiletime_to_timespec(&(fdata.ftLastWriteTime), &(buf->st_mtim));\n+\t\tfiletime_to_timespec(&(fdata.ftCreationTime), &(buf->st_ctim));\n \t\treturn 0;\n \n \tcase FILE_TYPE_CHAR:\ndiff --git a/compat/mingw.h b/compat/mingw.h\nindex f2a78b4..8dee9c9 100644\n--- a/compat/mingw.h\n+++ b/compat/mingw.h\n@@ -293,22 +293,48 @@ static inline long long filetime_to_hnsec(const FILETIME *ft)\n \treturn winTime - 116444736000000000LL;\n }\n \n-static inline time_t filetime_to_time_t(const FILETIME *ft)\n+struct timespec {\n+\ttime_t tv_sec;\n+\tlong tv_nsec;\n+};\n+\n+static inline void filetime_to_timespec(const FILETIME *ft, struct timespec *ts)\n {\n-\treturn (time_t)(filetime_to_hnsec(ft) / 10000000);\n+\tlong long hnsec = filetime_to_hnsec(ft);\n+\tts->tv_sec = (time_t)(hnsec / 10000000);\n+\tts->tv_nsec = (hnsec % 10000000) * 100;\n }\n \n /*\n- * Use mingw specific stat()/lstat()/fstat() implementations on Windows.\n+ * Use mingw specific stat()/lstat()/fstat() implementations on Windows,\n+ * including our own struct stat with 64 bit st_size and nanosecond-precision\n+ * file times.\n  */\n #define off_t off64_t\n #define lseek _lseeki64\n \n-/* use struct stat with 64 bit st_size */\n+struct mingw_stat {\n+    _dev_t st_dev;\n+    _ino_t st_ino;\n+    _mode_t st_mode;\n+    short st_nlink;\n+    short st_uid;\n+    short st_gid;\n+    _dev_t st_rdev;\n+    off64_t st_size;\n+    struct timespec st_atim;\n+    struct timespec st_mtim;\n+    struct timespec st_ctim;\n+};\n+\n+#define st_atime st_atim.tv_sec\n+#define st_mtime st_mtim.tv_sec\n+#define st_ctime st_ctim.tv_sec\n+\n #ifdef stat\n #undef stat\n #endif\n-#define stat _stati64\n+#define stat mingw_stat\n int mingw_lstat(const char *file_name, struct stat *buf);\n int mingw_stat(const char *file_name, struct stat *buf);\n int mingw_fstat(int fd, struct stat *buf);\n@@ -321,13 +347,6 @@ int mingw_fstat(int fd, struct stat *buf);\n #endif\n #define lstat mingw_lstat\n \n-#ifndef _stati64\n-# define _stati64(x,y) mingw_stat(x,y)\n-#elif defined (_USE_32BIT_TIME_T)\n-# define _stat32i64(x,y) mingw_stat(x,y)\n-#else\n-# define _stat64(x,y) mingw_stat(x,y)\n-#endif\n \n int mingw_utime(const char *file_name, const struct utimbuf *times);\n #define utime mingw_utime\ndiff --git a/config.mak.uname b/config.mak.uname\nindex b64b63c..a18a4cc 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -346,7 +346,7 @@ ifeq ($(uname_S),Windows)\n \tNO_SVN_TESTS = YesPlease\n \tRUNTIME_PREFIX = YesPlease\n \tNO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease\n-\tNO_NSEC = YesPlease\n+\tUSE_NSEC = YesPlease\n \tUSE_WIN32_MMAP = YesPlease\n \t# USE_NED_ALLOCATOR = YesPlease\n \tUNRELIABLE_FSTAT = UnfortunatelyYes\n@@ -498,7 +498,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n \tNO_PERL_MAKEMAKER = YesPlease\n \tRUNTIME_PREFIX = YesPlease\n \tNO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease\n-\tNO_NSEC = YesPlease\n+\tUSE_NSEC = YesPlease\n \tUSE_WIN32_MMAP = YesPlease\n \tUSE_NED_ALLOCATOR = YesPlease\n \tUNRELIABLE_FSTAT = UnfortunatelyYes\n-- \n2.3.0.3.ge7778af\n\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256008","messageId":"xmqqbnkysygl.fsf@gitster.dls.corp.google.com","threadId":"38553","inReplyTo":"54DBEAA5.6000205@gmail.com","subject":"Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-02-12T19:48:26Z","receivedAt":"2015-02-12T19:48:26Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Karsten Blees <karsten.blees@gmail.com> writes:\n\n> This patch series was inspired by the problem that Git does not\n> detect changed file content if st_size, st_mtime and st_ctime\n> are unchanged. This was apparently caused by VSS2Git resetting\n> mtime to a value in the past. [1]\n>\n> I believe (or rather hope) that all involved in the discussion\n> agree that Git cannot reasonably be expected to detect changed\n> file content if file time(s) are reset on purpose.\n>\n> However, some users have expressed concerns that 'same size and\n> mtime' [2] may theoretically happen by chance in daily operation.\n\nHmph.\n\nHaven't we already accepted that it is not just \"may theoretically\nhappen\" and had counter-measures in racy-git detection machinery\nfor quite some time?\n"},{"id":"256027","messageId":"db4e2b740bef943fb1adaf2b22893136@www.dscho.org","threadId":"38553","inReplyTo":"xmqqbnkysygl.fsf@gitster.dls.corp.google.com","subject":"Re: Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Johannes Schindelin","fromEmail":"johannes.schindelin@gmx.de","sentAt":"2015-02-12T22:30:51Z","receivedAt":"2015-02-12T22:30:51Z","isPatch":true,"sender":{"key":"johannes.schindelin@gmx.de","avatar":"https://avatars.githubusercontent.com/u/127790?v=4"},"body":"Hi,\n\nOn 2015-02-12 20:48, Junio C Hamano wrote:\n> Karsten Blees <karsten.blees@gmail.com> writes:\n> \n>> This patch series was inspired by the problem that Git does not\n>> detect changed file content if st_size, st_mtime and st_ctime\n>> are unchanged. This was apparently caused by VSS2Git resetting\n>> mtime to a value in the past. [1]\n>>\n>> I believe (or rather hope) that all involved in the discussion\n>> agree that Git cannot reasonably be expected to detect changed\n>> file content if file time(s) are reset on purpose.\n>>\n>> However, some users have expressed concerns that 'same size and\n>> mtime' [2] may theoretically happen by chance in daily operation.\n> \n> Hmph.\n> \n> Haven't we already accepted that it is not just \"may theoretically\n> happen\" and had counter-measures in racy-git detection machinery\n> for quite some time?\n\nI agree that the \"racy-git\" reference is more of a red herring; it appears that the report Karsten referred to is based on a different understanding of the mtime than mine (it is purported that the mtime of a file should reflect the time when it entered the repository, rather than the time when the file was written to disk).\n\nPlease let that not distract you.\n\nThis patch series has merits on the basis of populating st_mtim.tv_nsec already. We can provide this information in the common case (i.e. NTFS, *not* FAT), so we should.\n\nCiao,\nDscho\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256033","messageId":"54DD2FDA.7030604@gmail.com","threadId":"38553","inReplyTo":"xmqqbnkysygl.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-12T22:57:30Z","receivedAt":"2015-02-12T22:57:30Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"Am 12.02.2015 um 20:48 schrieb Junio C Hamano:\n> Karsten Blees <karsten.blees@gmail.com> writes:\n> \n>> However, some users have expressed concerns that 'same size and\n>> mtime' [2] may theoretically happen by chance in daily operation.\n> \n> Hmph.\n> \n> Haven't we already accepted that it is not just \"may theoretically\n> happen\" and had counter-measures in racy-git detection machinery\n> for quite some time?\n> \n\nRacy-git only triggers for files that are modified at the same time\nas .git/index (i.e. we don't know if the stat cache is up to date).\n\nThis is more about copying 'old' things around, which usually also\ncopies mtime on Windows. E.g.:\n\n  # create two files with slightly different mtime\n  for i in {1..10}; do (echo \"v1\" >> test); done &&\n  for i in {1..10}; do (echo \"v2\" >> test2); done\n  # wait a bit so that '.git/index' is always newer than 'test' / 'test2'\n  sleep 1\n  git add test\n  git commit -m v1\n  # copy test2 over test (similar to 'cp -p', but native 'copy' also\n  # copies mtime nanoseconds)\n  cmd //c \"copy /y test2 test\"\n  git add test\n  git commit -m v2\n\nWithout these patches, git does not detect the change, and the second\ngit add / git commit are noops.\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256038","messageId":"54DD340C.50204@virtuell-zuhause.de","threadId":"38553","inReplyTo":"54DBEB5C.5070804@gmail.com","subject":"Re: [PATCH 3/3] Win32: implement nanosecond-precision file times","fromName":"Thomas Braun","fromEmail":"thomas.braun@virtuell-zuhause.de","sentAt":"2015-02-12T23:15:24Z","receivedAt":"2015-02-12T23:15:24Z","isPatch":true,"sender":{"key":"thomas.braun@virtuell-zuhause.de","avatar":"https://avatars.githubusercontent.com/u/1185677?v=4"},"body":"Am 12.02.2015 um 00:53 schrieb Karsten Blees:\n> We no longer use any of MSVCRT's stat-functions, so there's no need to\n> stick to a CRT-compatible 'struct stat' either.\n> \n> Define and use our own POSIX-2013-compatible 'struct stat' with nanosecond-\n> precision file times.\n> \n> Signed-off-by: Karsten Blees <blees@dcon.de>\n> ---\n>  compat/mingw.c   | 12 ++++++------\n>  compat/mingw.h   | 43 +++++++++++++++++++++++++++++++------------\n>  config.mak.uname |  4 ++--\n>  3 files changed, 39 insertions(+), 20 deletions(-)\n> \n> diff --git a/compat/mingw.c b/compat/mingw.c\n> index 6d73a3d..e4d5e3f 100644\n> --- a/compat/mingw.c\n> +++ b/compat/mingw.c\n> @@ -442,9 +442,9 @@ static int do_lstat(int follow, const char *file_name, struct stat *buf)\n>  \t\tbuf->st_size = fdata.nFileSizeLow |\n>  \t\t\t(((off_t)fdata.nFileSizeHigh)<<32);\n>  \t\tbuf->st_dev = buf->st_rdev = 0; /* not used by Git */\n> -\t\tbuf->st_atime = filetime_to_time_t(&(fdata.ftLastAccessTime));\n> -\t\tbuf->st_mtime = filetime_to_time_t(&(fdata.ftLastWriteTime));\n> -\t\tbuf->st_ctime = filetime_to_time_t(&(fdata.ftCreationTime));\n> +\t\tfiletime_to_timespec(&(fdata.ftLastAccessTime), &(buf->st_atim));\n> +\t\tfiletime_to_timespec(&(fdata.ftLastWriteTime), &(buf->st_mtim));\n> +\t\tfiletime_to_timespec(&(fdata.ftCreationTime), &(buf->st_ctim));\n>  \t\tif (fdata.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) {\n>  \t\t\tWIN32_FIND_DATAW findbuf;\n>  \t\t\tHANDLE handle = FindFirstFileW(wfilename, &findbuf);\n> @@ -550,9 +550,9 @@ int mingw_fstat(int fd, struct stat *buf)\n>  \t\tbuf->st_mode = file_attr_to_st_mode(fdata.dwFileAttributes);\n>  \t\tbuf->st_size = fdata.nFileSizeLow |\n>  \t\t\t(((off_t)fdata.nFileSizeHigh)<<32);\n> -\t\tbuf->st_atime = filetime_to_time_t(&(fdata.ftLastAccessTime));\n> -\t\tbuf->st_mtime = filetime_to_time_t(&(fdata.ftLastWriteTime));\n> -\t\tbuf->st_ctime = filetime_to_time_t(&(fdata.ftCreationTime));\n> +\t\tfiletime_to_timespec(&(fdata.ftLastAccessTime), &(buf->st_atim));\n> +\t\tfiletime_to_timespec(&(fdata.ftLastWriteTime), &(buf->st_mtim));\n> +\t\tfiletime_to_timespec(&(fdata.ftCreationTime), &(buf->st_ctim));\n>  \t\treturn 0;\n>  \n>  \tcase FILE_TYPE_CHAR:\n> diff --git a/compat/mingw.h b/compat/mingw.h\n> index f2a78b4..8dee9c9 100644\n> --- a/compat/mingw.h\n> +++ b/compat/mingw.h\n> @@ -293,22 +293,48 @@ static inline long long filetime_to_hnsec(const FILETIME *ft)\n>  \treturn winTime - 116444736000000000LL;\n>  }\n>  \n> -static inline time_t filetime_to_time_t(const FILETIME *ft)\n> +struct timespec {\n> +\ttime_t tv_sec;\n> +\tlong tv_nsec;\n> +};\n> +\n> +static inline void filetime_to_timespec(const FILETIME *ft, struct timespec *ts)\n>  {\n> -\treturn (time_t)(filetime_to_hnsec(ft) / 10000000);\n> +\tlong long hnsec = filetime_to_hnsec(ft);\n> +\tts->tv_sec = (time_t)(hnsec / 10000000);\n> +\tts->tv_nsec = (hnsec % 10000000) * 100;\n>  }\n>  \n>  /*\n> - * Use mingw specific stat()/lstat()/fstat() implementations on Windows.\n> + * Use mingw specific stat()/lstat()/fstat() implementations on Windows,\n> + * including our own struct stat with 64 bit st_size and nanosecond-precision\n> + * file times.\n>   */\n>  #define off_t off64_t\n>  #define lseek _lseeki64\n>  \n> -/* use struct stat with 64 bit st_size */\n> +struct mingw_stat {\n> +    _dev_t st_dev;\n> +    _ino_t st_ino;\n> +    _mode_t st_mode;\n> +    short st_nlink;\n> +    short st_uid;\n> +    short st_gid;\n> +    _dev_t st_rdev;\n> +    off64_t st_size;\n> +    struct timespec st_atim;\n> +    struct timespec st_mtim;\n> +    struct timespec st_ctim;\n> +};\n> +\n> +#define st_atime st_atim.tv_sec\n> +#define st_mtime st_mtim.tv_sec\n> +#define st_ctime st_ctim.tv_sec\n> +\n>  #ifdef stat\n>  #undef stat\n>  #endif\n> -#define stat _stati64\n> +#define stat mingw_stat\n>  int mingw_lstat(const char *file_name, struct stat *buf);\n>  int mingw_stat(const char *file_name, struct stat *buf);\n>  int mingw_fstat(int fd, struct stat *buf);\n> @@ -321,13 +347,6 @@ int mingw_fstat(int fd, struct stat *buf);\n>  #endif\n>  #define lstat mingw_lstat\n>  \n> -#ifndef _stati64\n> -# define _stati64(x,y) mingw_stat(x,y)\n> -#elif defined (_USE_32BIT_TIME_T)\n> -# define _stat32i64(x,y) mingw_stat(x,y)\n> -#else\n> -# define _stat64(x,y) mingw_stat(x,y)\n> -#endif\n>  \n>  int mingw_utime(const char *file_name, const struct utimbuf *times);\n>  #define utime mingw_utime\n> diff --git a/config.mak.uname b/config.mak.uname\n> index b64b63c..a18a4cc 100644\n> --- a/config.mak.uname\n> +++ b/config.mak.uname\n> @@ -346,7 +346,7 @@ ifeq ($(uname_S),Windows)\n>  \tNO_SVN_TESTS = YesPlease\n>  \tRUNTIME_PREFIX = YesPlease\n>  \tNO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease\n> -\tNO_NSEC = YesPlease\n> +\tUSE_NSEC = YesPlease\n>  \tUSE_WIN32_MMAP = YesPlease\n>  \t# USE_NED_ALLOCATOR = YesPlease\n>  \tUNRELIABLE_FSTAT = UnfortunatelyYes\n> @@ -498,7 +498,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n>  \tNO_PERL_MAKEMAKER = YesPlease\n>  \tRUNTIME_PREFIX = YesPlease\n>  \tNO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease\n> -\tNO_NSEC = YesPlease\n> +\tUSE_NSEC = YesPlease\n>  \tUSE_WIN32_MMAP = YesPlease\n>  \tUSE_NED_ALLOCATOR = YesPlease\n>  \tUNRELIABLE_FSTAT = UnfortunatelyYes\n> \n\nWhy not also enable it in our special msysgit section?\n\ndiff --git a/config.mak.uname b/config.mak.uname\nindex b64b63c..6326794 100644\n--- a/config.mak.uname\n+++ b/config.mak.uname\n@@ -535,6 +535,7 @@ ifneq (,$(wildcard ../THIS_IS_MSYSGIT))\n        INTERNAL_QSORT = YesPlease\n        HAVE_LIBCHARSET_H = YesPlease\n        NO_GETTEXT = YesPlease\n+       USE_NSEC = YesPlease\n else\n        NO_CURL = YesPlease\n endif\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256041","messageId":"xmqqsieaog4a.fsf@gitster.dls.corp.google.com","threadId":"38553","inReplyTo":"54DD2FDA.7030604@gmail.com","subject":"Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-02-12T23:38:13Z","receivedAt":"2015-02-12T23:38:13Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Karsten Blees <karsten.blees@gmail.com> writes:\n\n> This is more about copying 'old' things around, which usually also\n> copies mtime on Windows. E.g.:\n>\n>   # create two files with slightly different mtime\n>   for i in {1..10}; do (echo \"v1\" >> test); done &&\n>   for i in {1..10}; do (echo \"v2\" >> test2); done\n>   # wait a bit so that '.git/index' is always newer than 'test' / 'test2'\n>   sleep 1\n>   git add test\n>   git commit -m v1\n>   # copy test2 over test (similar to 'cp -p', but native 'copy' also\n>   # copies mtime nanoseconds)\n>   cmd //c \"copy /y test2 test\"\n>   git add test\n>   git commit -m v2\n>\n> Without these patches, git does not detect the change, and the second\n> git add / git commit are noops.\n\nWe do have sec/nsec fields in cache_time structure, so I have\nnothing against updating the msysGit port to fill that value.\n\nI was and am just reacting to the fact that this is sold as if it\n\"fixes\" something.  It doesn't fundamentally change the fact that\nmtime that does not follow the semantics Dscho mentioned in his\nearlier message does not work well with Git.\n\nHaving said that, even with such a patch, as long as the system is\nsufficiently fast, test and test2 will have nonoseconds identical\ntimestamp and you would have the same issue, no?\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256042","messageId":"54DD3AE0.2050307@gmail.com","threadId":"38553","inReplyTo":"54DD340C.50204@virtuell-zuhause.de","subject":"Re: [PATCH 3/3] Win32: implement nanosecond-precision file times","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-12T23:44:32Z","receivedAt":"2015-02-12T23:44:32Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"Am 13.02.2015 um 00:15 schrieb Thomas Braun:\n> Am 12.02.2015 um 00:53 schrieb Karsten Blees:\n>> diff --git a/config.mak.uname b/config.mak.uname\n>> index b64b63c..a18a4cc 100644\n>> --- a/config.mak.uname\n>> +++ b/config.mak.uname\n>> @@ -346,7 +346,7 @@ ifeq ($(uname_S),Windows)\n>>  \tNO_SVN_TESTS = YesPlease\n>>  \tRUNTIME_PREFIX = YesPlease\n>>  \tNO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease\n>> -\tNO_NSEC = YesPlease\n>> +\tUSE_NSEC = YesPlease\n>>  \tUSE_WIN32_MMAP = YesPlease\n>>  \t# USE_NED_ALLOCATOR = YesPlease\n>>  \tUNRELIABLE_FSTAT = UnfortunatelyYes\n>> @@ -498,7 +498,7 @@ ifneq (,$(findstring MINGW,$(uname_S)))\n>>  \tNO_PERL_MAKEMAKER = YesPlease\n>>  \tRUNTIME_PREFIX = YesPlease\n>>  \tNO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease\n>> -\tNO_NSEC = YesPlease\n>> +\tUSE_NSEC = YesPlease\n>>  \tUSE_WIN32_MMAP = YesPlease\n>>  \tUSE_NED_ALLOCATOR = YesPlease\n>>  \tUNRELIABLE_FSTAT = UnfortunatelyYes\n>>\n> \n> Why not also enable it in our special msysgit section?\n> \n> diff --git a/config.mak.uname b/config.mak.uname\n> index b64b63c..6326794 100644\n> --- a/config.mak.uname\n> +++ b/config.mak.uname\n> @@ -535,6 +535,7 @@ ifneq (,$(wildcard ../THIS_IS_MSYSGIT))\n>         INTERNAL_QSORT = YesPlease\n>         HAVE_LIBCHARSET_H = YesPlease\n>         NO_GETTEXT = YesPlease\n> +       USE_NSEC = YesPlease\n>  else\n>         NO_CURL = YesPlease\n>  endif\n> \n\nThe msysgit section is within MINGW (i.e. already covered by the 2nd hunk), don't let the indentation fool you.\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256046","messageId":"54DD5A89.5080502@gmail.com","threadId":"38553","inReplyTo":"xmqqsieaog4a.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-13T01:59:37Z","receivedAt":"2015-02-13T01:59:37Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"Am 13.02.2015 um 00:38 schrieb Junio C Hamano:\n> Karsten Blees <karsten.blees@gmail.com> writes:\n> \n>> This is more about copying 'old' things around, which usually also\n>> copies mtime on Windows. E.g.:\n>>\n>>   # create two files with slightly different mtime\n>>   for i in {1..10}; do (echo \"v1\" >> test); done &&\n>>   for i in {1..10}; do (echo \"v2\" >> test2); done\n>>   # wait a bit so that '.git/index' is always newer than 'test' / 'test2'\n>>   sleep 1\n>>   git add test\n>>   git commit -m v1\n>>   # copy test2 over test (similar to 'cp -p', but native 'copy' also\n>>   # copies mtime nanoseconds)\n>>   cmd //c \"copy /y test2 test\"\n>>   git add test\n>>   git commit -m v2\n>>\n>> Without these patches, git does not detect the change, and the second\n>> git add / git commit are noops.\n> \n> We do have sec/nsec fields in cache_time structure, so I have\n> nothing against updating the msysGit port to fill that value.\n> \n> I was and am just reacting to the fact that this is sold as if it\n> \"fixes\" something.\n\nSorry, that must have been a misunderstanding. This series does\nNOT fix the problem with VSS2Git, nor any other tool that abuses\nmtime for the author's birthday or whatever.\n\nThe issue that two files may accidentally have the same size and\nmtime was just brought up in this discussion.\n\n> It doesn't fundamentally change the fact that\n> mtime that does not follow the semantics Dscho mentioned in his\n> earlier message does not work well with Git.\n> \n> Having said that, even with such a patch, as long as the system is\n> sufficiently fast, test and test2 will have nonoseconds identical\n> timestamp and you would have the same issue, no?\n> \n\nRight. Where \"sufficiently fast\" would mean opening and closing a\nfile ten times in less than 100ns...on Windows... ;-)\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256074","messageId":"xmqq61b5obla.fsf@gitster.dls.corp.google.com","threadId":"38553","inReplyTo":"54DD5A89.5080502@gmail.com","subject":"Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-02-13T19:28:17Z","receivedAt":"2015-02-13T19:28:17Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Karsten Blees <karsten.blees@gmail.com> writes:\n\n> Am 13.02.2015 um 00:38 schrieb Junio C Hamano:\n>> \n>> We do have sec/nsec fields in cache_time structure, so I have\n>> nothing against updating the msysGit port to fill that value.\n\nHaving said that, we do not enable the NSEC stuff by default on Unix\nfor a reason.  I'd expect those who know Windows filesystems well to\npick the default there wisely ;-)\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256168","messageId":"54E250A6.4070605@gmail.com","threadId":"38553","inReplyTo":"xmqq61b5obla.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-16T20:18:46Z","receivedAt":"2015-02-16T20:18:46Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"Am 13.02.2015 um 20:28 schrieb Junio C Hamano:\n> Karsten Blees <karsten.blees@gmail.com> writes:\n> \n>> Am 13.02.2015 um 00:38 schrieb Junio C Hamano:\n>>>\n>>> We do have sec/nsec fields in cache_time structure, so I have\n>>> nothing against updating the msysGit port to fill that value.\n> \n> Having said that, we do not enable the NSEC stuff by default on Unix\n> for a reason.  I'd expect those who know Windows filesystems well to\n> pick the default there wisely ;-)\n> \n\nNow I'm a bit confused about the discrepancy between racy-git.txt and\nthe Makefile.\n\nRacy-git.txt explains that the nsec-part may be dropped when an inode\nis flushed to disk if the file system doesn't support nsec resolution.\nThis was supposedly an issue with the Linux kernel fixed back in 2005.\n\nIn my understanding, this means that git would see the file as\nchanged and re-check the content (i.e. it will hurt performance).\n\nIOW: Git may be slow if the file system cache has better file time\nresolution than the on-disk file system representation.\n\n\nHowever, the Makefile has this to say on the subject:\n\n# Define USE_NSEC below if you want git to care about sub-second file mtimes\n# and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and\n# it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely\n# randomly break unless your underlying filesystem supports those sub-second\n# times (my ext3 doesn't).\n\nAm I missing something? Is there anything in Git that will actually\n\"break\" with USE_NSEC if the OS / file system doesn't support it\n(rather than just being slow)?\n\nHistory:\n* The Makefile comment was added in 2005 (bdd4da59), along with a\n  comment in read-cache.c explaining the issue (i.e. flushing to disk\n  will clear the nsec field).\n* The comment in read-cache.c was removed in 2008 (7a51ed66),\n  seemingly dropping USE_NSEC support entirely.\n* USE_NSEC support was re-added (without the read-cache.c comment) in\n  2009 (fba2f38a).\n\n\nRegarding the Windows situation: I've just verified (on my Win7 x64\nbox) that file times obtained through a variety of APIs (GetFileTime,\nGetFileAttributesEx, GetFileInformationByHandle, FindFirstFile) are\nconsistent and properly rounded to the file system's resolution (e.g.\n10ms / 2s for FAT). This is even if the file is still open and I try\nto SetFileTime() to unrounded values.\n\nSo I think enabling USE_NSEC should be fine on Windows.\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256170","messageId":"xmqqtwyl4idp.fsf@gitster.dls.corp.google.com","threadId":"38553","inReplyTo":"54E250A6.4070605@gmail.com","subject":"Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2015-02-16T22:10:58Z","receivedAt":"2015-02-16T22:10:58Z","isPatch":true,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Karsten Blees <karsten.blees@gmail.com> writes:\n\n> However, the Makefile has this to say on the subject:\n>\n> # Define USE_NSEC below if you want git to care about sub-second file mtimes\n> # and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and\n> # it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely\n> # randomly break unless your underlying filesystem supports those sub-second\n> # times (my ext3 doesn't).\n>\n> Am I missing something?\n\nI think \"it would break\" is about show-diff which wanted to use the\ncached stat information for freshness.\n\n\t>foo\n\tgit update-index --add foo\n        sleep 2\n        >foo\n        git diff-files ;# modern counterpart of show-diff\n\nwould say that \"foo\" is *different*, because the plumbing commands\nlike diff-files expect you to refresh the index before you call\nthem.\n\nAnd if you did \"git update-index --refresh\" after touching \"foo\" the\nlast time before running \"git diff-files\" in the above sequence, you\nshould expect that it does not say \"foo\" is different, no matter how\nmuch time passes between the time you run that \"refresh\" and\n\"diff-files\" (or between the time you last touched \"foo\" and you run\n\"refresh\", for that matter), as long as you do not touch \"foo\" in\nthe meantime.  The following should say \"foo\" is *not* different,\nthat is:\n\n\t>foo\n\tgit update-index --add foo\n        sleep 2\n        >foo\n        sleep arbitrary\n        git update-index --refresh\n        sleep arbitrary\n        git diff-files ;# modern counterpart of show-diff\n\nIf you use NSEC, however, and \"refresh\" grabbed a subsecond time and\nthen later \"diff-files\" learned a truncated/rounded time because the\nfilesystem later purged the cached inodes and re-read it from the\nunderlying filesystem with no subsecond time resolution, the times\nwould not match so you will again see \"diff-files\" report that \"foo\"\nis now different.\n\nThat is what the comment you cited is about.\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"},{"id":"256241","messageId":"54E3B959.9040105@gmail.com","threadId":"38553","inReplyTo":"xmqqtwyl4idp.fsf@gitster.dls.corp.google.com","subject":"Re: [PATCH 0/3] Win32: nanosecond-precision file times","fromName":"Karsten Blees","fromEmail":"karsten.blees@gmail.com","sentAt":"2015-02-17T21:57:45Z","receivedAt":"2015-02-17T21:57:45Z","isPatch":true,"sender":{"key":"karsten.blees@gmail.com","avatar":"https://avatars.githubusercontent.com/u/1111200?v=4"},"body":"Am 16.02.2015 um 23:10 schrieb Junio C Hamano:\n> Karsten Blees <karsten.blees@gmail.com> writes:\n> \n>> However, the Makefile has this to say on the subject:\n>>\n>> # Define USE_NSEC below if you want git to care about sub-second file mtimes\n>> # and ctimes. Note that you need recent glibc (at least 2.2.4) for this, and\n>> # it will BREAK YOUR LOCAL DIFFS! show-diff and anything using it will likely\n>> # randomly break unless your underlying filesystem supports those sub-second\n>> # times (my ext3 doesn't).\n>>\n>> Am I missing something?\n> \n[...]\n> \n> If you use NSEC, however, and \"refresh\" grabbed a subsecond time and\n> then later \"diff-files\" learned a truncated/rounded time because the\n> filesystem later purged the cached inodes and re-read it from the\n> underlying filesystem with no subsecond time resolution, the times\n> would not match so you will again see \"diff-files\" report that \"foo\"\n> is now different.\n> \n> That is what the comment you cited is about.\n> \n\nOK, so it all boils down to the \"inode cache doesn't round to on-disk\nresolution\" issue after all, as explained in racy-git.txt.\n\nBut then the Makefile comment is quite misleading. Enabling USE_NSEC\nwill not unconditionally \"BREAK YOUR LOCAL DIFFS\". Show-diff / diff-files\nwill also not \"break\", but may report false positives instead (which may\nbe worse than failing hard...).\n\nIt also seems to me that this is a Linux-only issue which is only remotely\nrelated to the USE_NSEC setting or file systems' timestamp resolutions.\n\nThe kernel patch referenced in racy-git.txt only addresses sub-second\nresolutions. So even if USE_NSEC is *disabled*, the diff-files issue will\nbite you on e.g. FAT32-formatted flash-drives on Linux, at least on\nre-mount (\"sync && echo 2>/proc/sys/vm/drop_caches\" didn't seem to trigger\nthe rounding, though).\n\nI also suspect that the sub-second rounding function of that patch\n(timespec_trunc()) takes some invalid shortcuts - if you configure the\nkernel for 300 jiffies per second (i.e. 3,333,333 ns per tick), UDF, NTFS,\nSMBFS and CIFS file times will most likely not be properly rounded in the\ninode cache. Haven't tested this, though.\n\nSo the only file systems with reliable file times on Linux seem to be\nthose with exactly 1s or 1ns resolution...?\n\n-- \n-- \n*** Please reply-to-all at all times ***\n*** (do not pretend to know who is subscribed and who is not) ***\n*** Please avoid top-posting. ***\nThe msysGit Wiki is here: https://github.com/msysgit/msysgit/wiki - Github accounts are free.\n\nYou received this message because you are subscribed to the Google\nGroups \"msysGit\" group.\nTo post to this group, send email to msysgit@googlegroups.com\nTo unsubscribe from this group, send email to\nmsysgit+unsubscribe@googlegroups.com\nFor more options, and view previous threads, visit this group at\nhttp://groups.google.com/group/msysgit?hl=en_US?hl=en\n\n--- \nYou received this message because you are subscribed to the Google Groups \"Git for Windows\" group.\nTo unsubscribe from this group and stop receiving emails from it, send an email to msysgit+unsubscribe@googlegroups.com.\nFor more options, visit https://groups.google.com/d/optout.\n"}]}