# [PATCH] t4130: work around Windows limitation 8 messages from 2016-07-27 to 2016-08-04. Participants: Johannes Schindelin, Johannes Sixt, Junio C Hamano. Thread: https://gitlist.dev/t/42940 ## Johannes Schindelin, 2016-07-27 12:39 Subject: [PATCH] t4130: work around Windows limitation Message-ID: <5bd59ca2f87e388350f3c8fb17c9a287661cd055.1469623136.git.johannes.schindelin@gmx.de> URL: https://gitlist.dev/e/5bd59ca2f87e388350f3c8fb17c9a287661cd055.1469623136.git.johannes.schindelin%40gmx.de ``` On Windows, it is already pretty expensive to try to recreate the stat() data that Git assumes is cheap to obtain. To make things halfway decent in performance, we even have to skip emulating the inode and to determine the number of hard links. This is not a huge problem, usually, as either the size or the mtime or the ctime are tell-tale enough to say when a file has changed, and even if not, those changes are typically made after the index file was written, triggering a rehashing of the files' contents. The t4130-apply-criss-cross-rename test case, however, requires the inode to determine that files of equal size were swapped, as renaming files does not update their mtime. And even if we use nanosecond-precision mtimes on Windows, the file system's time granularity is typically much coarser (100ms for NTFS, 2s for FAT). That means that every once in a while, t4130 fails on Windows. This patch provides the work-around by pretending that the index file was written earlier than it actually was. Signed-off-by: Johannes Schindelin --- Published-As: https://github.com/dscho/git/releases/tag/t4130-mingw-v1 t/t4130-apply-criss-cross-rename.sh | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh index d173acd..ee91af8 100755 --- a/t/t4130-apply-criss-cross-rename.sh +++ b/t/t4130-apply-criss-cross-rename.sh @@ -29,6 +29,14 @@ test_expect_success 'criss-cross rename' ' ' test_expect_success 'diff -M -B' ' + if test_have_prereq MINGW + then + # On Windows it is prohbitively expensive to retrieve the + # equivalent of an "inode" when calling stat(), therefore we + # rely on mtime/ctime/size changes to let us know whether a + # file has changed, including the mtime relative to the index. + test-chmtime -1 .git/index + fi && git diff -M -B > diff && git reset --hard @@ -52,6 +60,14 @@ test_expect_success 'criss-cross rename' ' ' test_expect_success 'diff -M -B' ' + if test_have_prereq MINGW + then + # On Windows it is prohbitively expensive to retrieve the + # equivalent of an "inode" when calling stat(), therefore we + # rely on mtime/ctime/size changes to let us know whether a + # file has changed, including the mtime relative to the index. + test-chmtime -1 .git/index + fi && git diff -M -B > diff && git reset --hard ' -- 2.9.0.281.g286a8d9 base-commit: 8c6d1f9807c67532e7fb545a944b064faff0f70b ``` ## Johannes Sixt, 2016-07-27 16:37 Subject: Re: [PATCH] t4130: work around Windows limitation Message-ID: <9d167448-84ce-e368-0bb4-41aa4452268d@kdbg.org> URL: https://gitlist.dev/e/9d167448-84ce-e368-0bb4-41aa4452268d%40kdbg.org In-Reply-To: <5bd59ca2f87e388350f3c8fb17c9a287661cd055.1469623136.git.johannes.schindelin@gmx.de> ``` Am 27.07.2016 um 14:39 schrieb Johannes Schindelin: > On Windows, it is already pretty expensive to try to recreate the stat() > data that Git assumes is cheap to obtain. To make things halfway decent > in performance, we even have to skip emulating the inode and to > determine the number of hard links. > > This is not a huge problem, usually, as either the size or the mtime or > the ctime are tell-tale enough to say when a file has changed, and even > if not, those changes are typically made after the index file was > written, triggering a rehashing of the files' contents. > > The t4130-apply-criss-cross-rename test case, however, requires the > inode to determine that files of equal size were swapped, as renaming > files does not update their mtime. And even if we use > nanosecond-precision mtimes on Windows, the file system's time > granularity is typically much coarser (100ms for NTFS, 2s for FAT). > > That means that every once in a while, t4130 fails on Windows. > > This patch provides the work-around by pretending that the index file > was written earlier than it actually was. > > Signed-off-by: Johannes Schindelin > --- > Published-As: https://github.com/dscho/git/releases/tag/t4130-mingw-v1 > t/t4130-apply-criss-cross-rename.sh | 16 ++++++++++++++++ > 1 file changed, 16 insertions(+) > > diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh > index d173acd..ee91af8 100755 > --- a/t/t4130-apply-criss-cross-rename.sh > +++ b/t/t4130-apply-criss-cross-rename.sh > @@ -29,6 +29,14 @@ test_expect_success 'criss-cross rename' ' > ' > > test_expect_success 'diff -M -B' ' > + if test_have_prereq MINGW > + then > + # On Windows it is prohbitively expensive to retrieve the > + # equivalent of an "inode" when calling stat(), therefore we > + # rely on mtime/ctime/size changes to let us know whether a > + # file has changed, including the mtime relative to the index. > + test-chmtime -1 .git/index > + fi && > git diff -M -B > diff && > git reset --hard > > @@ -52,6 +60,14 @@ test_expect_success 'criss-cross rename' ' > ' > > test_expect_success 'diff -M -B' ' > + if test_have_prereq MINGW > + then > + # On Windows it is prohbitively expensive to retrieve the > + # equivalent of an "inode" when calling stat(), therefore we > + # rely on mtime/ctime/size changes to let us know whether a > + # file has changed, including the mtime relative to the index. > + test-chmtime -1 .git/index > + fi && > git diff -M -B > diff && > git reset --hard > ' > How about this instead? It makes the sizes of the files different, which is a bit less obscure, IMHO. ---- 8< ---- [PATCH] t4130: work around Windows limitation On Windows, it is already pretty expensive to try to recreate the stat() data that Git assumes is cheap to obtain. To make things halfway decent in performance, we even have to skip emulating the inode and to determine the number of hard links. This is not a huge problem, usually, as either the size or the mtime or the ctime are tell-tale enough to say when a file has changed, and even if not, those changes are typically made after the index file was written, triggering a rehashing of the files' contents. The t4130-apply-criss-cross-rename test case, however, requires the inode to determine that files of equal size were swapped, as renaming files does not update their mtime. Every once in a while, t4130 fails on Windows of this missing piece. Equal file sizes are not crucial for the test cases, however. Hence, generate files with different sizes so that there is some property that the swapped files can be discovered reliably even on Windows. Helped-by: Johannes Schindelin Signed-off-by: Johannes Sixt --- t/t4130-apply-criss-cross-rename.sh | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh index d173acd..f8a313b 100755 --- a/t/t4130-apply-criss-cross-rename.sh +++ b/t/t4130-apply-criss-cross-rename.sh @@ -13,9 +13,13 @@ create_file() { } test_expect_success 'setup' ' - create_file file1 "File1 contents" && - create_file file2 "File2 contents" && - create_file file3 "File3 contents" && + # Ensure that file sizes are different, because on Windows + # lstat() does not discover inode numbers, and we need + # other properties to discover swapped files + # (mtime is not always different, either). + create_file file1 "some content" && + create_file file2 "some other content" && + create_file file3 "again something else" && git add file1 file2 file3 && git commit -m 1 ' -- 2.9.0.443.ga8520ad ``` ## Junio C Hamano, 2016-07-27 16:40 Subject: Re: [PATCH] t4130: work around Windows limitation Message-ID: URL: https://gitlist.dev/e/CAPc5daW10y_5rat0Au-1NWXUf2QYFrDfQeprqq9NsYPBjdqOTw%40mail.gmail.com In-Reply-To: <9d167448-84ce-e368-0bb4-41aa4452268d@kdbg.org> ``` On Wed, Jul 27, 2016 at 9:37 AM, Johannes Sixt wrote: > > How about this instead? It makes the sizes of the files different, > which is a bit less obscure, IMHO. > ... > Equal file sizes are not crucial for the test cases, however. That does sounds like a better solution. ``` ## Johannes Schindelin, 2016-07-30 08:11 Subject: Re: [PATCH] t4130: work around Windows limitation Message-ID: URL: https://gitlist.dev/e/alpine.DEB.2.20.1607301011140.11824%40virtualbox In-Reply-To: ``` Hi, On Wed, 27 Jul 2016, Junio C Hamano wrote: > On Wed, Jul 27, 2016 at 9:37 AM, Johannes Sixt wrote: > > > > How about this instead? It makes the sizes of the files different, > > which is a bit less obscure, IMHO. > > ... > > Equal file sizes are not crucial for the test cases, however. > > That does sounds like a better solution. Fine by me. Thanks, Dscho ``` ## Johannes Sixt, 2016-08-03 06:15 Subject: [PATCH v2] t4130: work around Windows limitation Message-ID: <244e3458-4a41-49bf-fba2-71da872e7f66@kdbg.org> URL: https://gitlist.dev/e/244e3458-4a41-49bf-fba2-71da872e7f66%40kdbg.org In-Reply-To: ``` On Windows, it is already pretty expensive to try to recreate the stat() data that Git assumes is cheap to obtain. To make things halfway decent in performance, we even have to skip emulating the inode and to determine the number of hard links. This is not a huge problem, usually, as either the size or the mtime or the ctime are tell-tale enough to say when a file has changed, and even if not, those changes are typically made after the index file was written, triggering a rehashing of the files' contents. The t4130-apply-criss-cross-rename test case, however, requires the inode to determine that files of equal size were swapped, as renaming files does not update their mtime. Every once in a while, t4130 fails on Windows because of this missing piece. Equal file sizes are not crucial for the test cases, however. Hence, generate files with different sizes so that there is some property that the swapped files can be discovered reliably even on Windows. Helped-by: Johannes Schindelin Signed-off-by: Johannes Sixt --- This fell through the cracks, I think. I marked it as v2 because there is a minor fixup in the commit message. t/t4130-apply-criss-cross-rename.sh | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh index d173acd..f8a313b 100755 --- a/t/t4130-apply-criss-cross-rename.sh +++ b/t/t4130-apply-criss-cross-rename.sh @@ -13,9 +13,13 @@ create_file() { } test_expect_success 'setup' ' - create_file file1 "File1 contents" && - create_file file2 "File2 contents" && - create_file file3 "File3 contents" && + # Ensure that file sizes are different, because on Windows + # lstat() does not discover inode numbers, and we need + # other properties to discover swapped files + # (mtime is not always different, either). + create_file file1 "some content" && + create_file file2 "some other content" && + create_file file3 "again something else" && git add file1 file2 file3 && git commit -m 1 ' -- 2.9.0.443.ga8520ad ``` ## Johannes Schindelin, 2016-08-03 12:03 Subject: Re: [PATCH v2] t4130: work around Windows limitation Message-ID: URL: https://gitlist.dev/e/alpine.DEB.2.20.1608031401370.79248%40virtualbox In-Reply-To: <244e3458-4a41-49bf-fba2-71da872e7f66@kdbg.org> ``` Hi Hannes, On Wed, 3 Aug 2016, Johannes Sixt wrote: > This fell through the cracks, I think. I marked it as v2 because > there is a minor fixup in the commit message. It appears that it did fall through the cracks. Given that it would make my life substantially easier (because I run the test suite often enough to hit this edge case multiple times a day), I would like to ask to fast-track it into v2.9.3. Thanks, Dscho ``` ## Junio C Hamano, 2016-08-03 15:50 Subject: Re: [PATCH v2] t4130: work around Windows limitation Message-ID: URL: https://gitlist.dev/e/xmqq4m71c0mz.fsf%40gitster.mtv.corp.google.com In-Reply-To: <244e3458-4a41-49bf-fba2-71da872e7f66@kdbg.org> ``` Johannes Sixt writes: > This fell through the cracks, I think. I marked it as v2 because > there is a minor fixup in the commit message. Thanks. The patch itself seems to got whitespace damaged somewhere between you and me, which I fixed up, but there may be similar damage to the proposed log message as well. What I read may already have whitespace damaged but I read the message twice and it made sense to me, so let's assume all is well ;-) Thanks. > > t/t4130-apply-criss-cross-rename.sh | 10 +++++++--- > 1 file changed, 7 insertions(+), 3 deletions(-) > > diff --git a/t/t4130-apply-criss-cross-rename.sh b/t/t4130-apply-criss-cross-rename.sh > index d173acd..f8a313b 100755 > --- a/t/t4130-apply-criss-cross-rename.sh > +++ b/t/t4130-apply-criss-cross-rename.sh > @@ -13,9 +13,13 @@ create_file() { > } > test_expect_success 'setup' ' > - create_file file1 "File1 contents" && > - create_file file2 "File2 contents" && > - create_file file3 "File3 contents" && > + # Ensure that file sizes are different, because on Windows > + # lstat() does not discover inode numbers, and we need > + # other properties to discover swapped files > + # (mtime is not always different, either). > + create_file file1 "some content" && > + create_file file2 "some other content" && > + create_file file3 "again something else" && > git add file1 file2 file3 && > git commit -m 1 > ' ``` ## Johannes Sixt, 2016-08-04 05:31 Subject: Re: [PATCH v2] t4130: work around Windows limitation Message-ID: URL: https://gitlist.dev/e/d00bfb15-a510-834a-9569-fa3fdb2db62c%40kdbg.org In-Reply-To: ``` Am 03.08.2016 um 17:50 schrieb Junio C Hamano: > Johannes Sixt writes: > The patch itself seems to got whitespace damaged somewhere > between you and me, which I fixed up, Sorry for the damaged patch. I forgot that Thunderbird's "Send again" feature as well as copying and pasting between Thunderbird windows is poison for patch text. What you have queued in js/t4130-rename-without-ino is correct. Thank you very much! -- Hannes ```