{"thread":{"id":"17758","subject":"not all tests pass when on an NFS partition","startedAt":"2009-02-12T20:20:19Z","lastAt":"2009-02-17T16:32:23Z","messageCount":7,"participants":["E R","Junio C Hamano","Brandon Casey"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"104407","messageId":"3a69fa7c0902121220n4ea2f19di6fe984dd2002772@mail.gmail.com","threadId":"17758","inReplyTo":null,"subject":"not all tests pass when on an NFS partition","fromName":"E R","fromEmail":"pc88mxer@gmail.com","sentAt":"2009-02-12T20:20:19Z","receivedAt":"2009-02-12T20:20:19Z","isPatch":false,"sender":{"key":"pc88mxer@gmail.com","avatar":null},"body":"When I run 'make test' for git-1.6.1.3 it will fail on this test when\nrunning on an NFS partition.\nIs this a known problem? Is this a real problem?\n\nMy uname -a: Linux XXX 2.6.18-92.1.17.el5 #1 SMP Tue Nov 4 13:45:01\nEST 2008 i686 athlon i386 GNU/Linux\n\n\n*** t5304-prune.sh ***\n* ok 1: setup\n* ok 2: prune stale packs\n* ok 3: prune --expire\n* FAIL 4: gc: implicit prune --expire\n\n\n before=$(git count-objects | sed \"s/ .*//\") &&\n BLOB=$(echo aleph_0 | git hash-object -w --stdin) &&\n BLOB_FILE=.git/objects/$(echo $BLOB | sed \"s/^../&\\//\") &&\n test $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n test -f $BLOB_FILE &&\n test-chmtime =-$((86400*14-30)) $BLOB_FILE &&\n git gc &&\n test $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n test -f $BLOB_FILE &&\n test-chmtime =-$((86400*14+1)) $BLOB_FILE &&\n git gc &&\n test $before = $(git count-objects | sed \"s/ .*//\") &&\n ! test -f $BLOB_FILE\n"},{"id":"104428","messageId":"7vk57vjqsd.fsf@gitster.siamese.dyndns.org","threadId":"17758","inReplyTo":"3a69fa7c0902121220n4ea2f19di6fe984dd2002772@mail.gmail.com","subject":"Re: not all tests pass when on an NFS partition","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2009-02-12T21:20:34Z","receivedAt":"2009-02-12T21:20:34Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"E R writes:\n\n> When I run 'make test' for git-1.6.1.3 it will fail on this test when\n> running on an NFS partition.\n\n... when the filesystem clock and the machine clock are out of sync.\n\n> Is this a known problem? Is this a real problem?\n\nIt's a problem whose cause can be easily guessed ;-)\n\nThe tests check if the default expiration period (14 days) are honored by\ncreating unreferenced objects that are slightly older and slightly newer\nthan the cut-off timestamp and running the gc. I suspect that the issue\nwith network filesystems whose clock are not grossly skewed can easily be\nworked around by tweaking these -30 and +1 constants to larger (but not\ntoo large to make the tests meaningless) values.\n\n> My uname -a: Linux XXX 2.6.18-92.1.17.el5 #1 SMP Tue Nov 4 13:45:01\n> EST 2008 i686 athlon i386 GNU/Linux\n>\n>\n> *** t5304-prune.sh ***\n> * ok 1: setup\n> * ok 2: prune stale packs\n> * ok 3: prune --expire\n> * FAIL 4: gc: implicit prune --expire\n>\n>\n> before=$(git count-objects | sed \"s/ .*//\") &&\n> BLOB=$(echo aleph_0 | git hash-object -w --stdin) &&\n> BLOB_FILE=.git/objects/$(echo $BLOB | sed \"s/^../&\\//\") &&\n> test $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n> test -f $BLOB_FILE &&\n> test-chmtime =-$((86400*14-30)) $BLOB_FILE &&\n> git gc &&\n> test $((1 + $before)) = $(git count-objects | sed \"s/ .*//\") &&\n> test -f $BLOB_FILE &&\n> test-chmtime =-$((86400*14+1)) $BLOB_FILE &&\n> git gc &&\n> test $before = $(git count-objects | sed \"s/ .*//\") &&\n> ! test -f $BLOB_FILE\n"},{"id":"104466","messageId":"Aq-rucZsypJQU8SHtTE8X62cEZZn-xSpOOIfA-I5czdtCkm2IyMjDw@cipher.nrlssc.navy.mil","threadId":"17758","inReplyTo":"7vk57vjqsd.fsf@gitster.siamese.dyndns.org","subject":"Re: not all tests pass when on an NFS partition","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2009-02-13T00:54:30Z","receivedAt":"2009-02-13T00:54:30Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"Junio C Hamano wrote:\n> E R writes:\n> \n>> When I run 'make test' for git-1.6.1.3 it will fail on this test when\n>> running on an NFS partition.\n> \n> ... when the filesystem clock and the machine clock are out of sync.\n\nI previously modified t5304 to make test-chmtime modify the mtime of\nthe file based on the system time rather than the file mtime precisely to\navoid this problem. i.e. the argument to test-chmtime is now prefixed with '='\n\nSo I don't think it is related to out of sync clock on NFS server and client.\nThere may be a bug in the NFS client though.\n\n>> Is this a known problem? Is this a real problem?\n\nKnown by me, but I haven't mentioned it since I haven't had a moment to\ndiagnose it.\n\n>> My uname -a: Linux XXX 2.6.18-92.1.17.el5 #1 SMP Tue Nov 4 13:45:01\n>> EST 2008 i686 athlon i386 GNU/Linux\n\nRHEL 5.2? me too.\n\nRHEL 4.7 works fine, same NFS server (running 5.2).\n\n-brandon\n"},{"id":"104821","messageId":"3a69fa7c0902151219y270681f7r17a108fdf44213ad@mail.gmail.com","threadId":"17758","inReplyTo":"Aq-rucZsypJQU8SHtTE8X62cEZZn-xSpOOIfA-I5czdtCkm2IyMjDw@cipher.nrlssc.navy.mil","subject":"Re: not all tests pass when on an NFS partition","fromName":"E R","fromEmail":"pc88mxer@gmail.com","sentAt":"2009-02-15T20:19:26Z","receivedAt":"2009-02-15T20:19:26Z","isPatch":false,"sender":{"key":"pc88mxer@gmail.com","avatar":null},"body":"On Thu, Feb 12, 2009 at 6:54 PM, Brandon Casey wrote:\n\n> RHEL 5.2? me too.\n>\n> RHEL 4.7 works fine, same NFS server (running 5.2).\n\nYes - it's Centos 5.2. Very interesting.\n"},{"id":"105157","messageId":"3a69fa7c0902170641m2ba9b8aeo950c9e3868ba6244@mail.gmail.com","threadId":"17758","inReplyTo":"3a69fa7c0902151219y270681f7r17a108fdf44213ad@mail.gmail.com","subject":"Re: not all tests pass when on an NFS partition","fromName":"E R","fromEmail":"pc88mxer@gmail.com","sentAt":"2009-02-17T14:41:30Z","receivedAt":"2009-02-17T14:41:30Z","isPatch":false,"sender":{"key":"pc88mxer@gmail.com","avatar":null},"body":"Btw - what kind of risks are involved in using git when test\nt5304-prune.sh doesn't pass? Do we risk repository corruption?\n\nOn Sun, Feb 15, 2009 at 2:19 PM, E R wrote:\n> On Thu, Feb 12, 2009 at 6:54 PM, Brandon Casey wrote:\n>\n>> RHEL 5.2? me too.\n>>\n>> RHEL 4.7 works fine, same NFS server (running 5.2).\n>\n> Yes - it's Centos 5.2. Very interesting.\n>\n"},{"id":"105170","messageId":"oguD57F0nZjzoh_EcoiIq6sB9GB3-tLhSNMQLqx9Xlh_z-uW_d0_yg@cipher.nrlssc.navy.mil","threadId":"17758","inReplyTo":"3a69fa7c0902151219y270681f7r17a108fdf44213ad@mail.gmail.com","subject":"Re: not all tests pass when on an NFS partition","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2009-02-17T16:16:04Z","receivedAt":"2009-02-17T16:16:04Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"E R wrote:\n> On Thu, Feb 12, 2009 at 6:54 PM, Brandon Casey wrote:\n> \n>> RHEL 5.2? me too.\n>>\n>> RHEL 4.7 works fine, same NFS server (running 5.2).\n> \n> Yes - it's Centos 5.2. Very interesting.\n\n\nIt looks like mtime (and ctime) are being filled in with the atime value.\n\nI modified builtin-prune.c as follows (warning: copy/pasted):\n\ndiff --git a/builtin-prune.c b/builtin-prune.c\nindex 545e9c1..2c1d351 100644\n--- a/builtin-prune.c\n+++ b/builtin-prune.c\n@@ -38,8 +38,17 @@ static int prune_object(char *path, const char *filename, con\n struct stat st;\n if (lstat(fullpath, &st))\n return error(\"Could not stat '%s'\", fullpath);\n- if (st.st_mtime > expire)\n+ printf(\"%s: expire %lu\\n\"\n+ \" inode %lu\\n\"\n+ \" atime %lu\\n\"\n+ \" mtime %lu\\n\"\n+ \" ctime %lu\\n\",\n+ fullpath, expire,\n+ st.st_ino, st.st_atime, st.st_mtime, st.st_ctime);\n+ if (st.st_mtime > expire) {\n+ puts(\"not expiring\");\n return 0;\n+ }\n }\n if (show_only || verbose) {\n enum object_type type = sha1_object_info(sha1, NULL);\n\n\nAfter inserting an 'exit' statement before test 4 of t5304, the second\n'git gc' prints the following:\n\n $ git gc\n Counting objects: 3, done.\n Writing objects: 100% (3/3), done.\n Total 3 (delta 0), reused 3 (delta 0)\n .git/objects/fb/f6228a25d50b1ea329e06e75b2f3f1de3793a0: expire 1233599127\n inode 46186578\n atime 1234807371\n mtime 1234807371\n ctime 1234807371\n not expiring\n\nAnd stat gives:\n\n $ stat .git/objects/fb/f6228a25d50b1ea329e06e75b2f3f1de3793a0\n File: `.git/objects/fb/f6228a25d50b1ea329e06e75b2f3f1de3793a0'\n Size: 23 Blocks: 16 IO Block: 4096 regular file\n Device: 1ah/26d Inode: 46186578 Links: 1\n Access: (0444/-r--r--r--) Uid: ( XXX) Gid: ( XXX)\n Access: 2009-02-16 12:02:51.000000000 -0600\n Modify: 2009-02-02 12:03:45.000000000 -0600\n Change: 2009-02-16 12:03:46.000000000 -0600\n\nAnd for the lazy\n\n $ perl -e 'print scalar(localtime(1234807371)), \"\\n\";'\n Mon Feb 16 12:02:51 2009\n\n\nCompiling with -O0 produces the same results.\nAlso, I am unable to write a program to reproduce this behavior.\n\nHere's a variation for you. 'git prune --expire 2.weeks.ago' only fails if\ncalled after a failed 'git gc'.\n\n# edit t5304-prune.sh to replace the second 'git gc' in test 4 with\n# 'git prune --expire 2.weeks.ago'\n$ rm -rf 'trash directory.t5304-prune'\n$ ./t5304-prune.sh\n# Success for test 4, but the new 11 and 12 still fail.\n# Plus, the correct values for atime, mtime, and ctime are printed out by\n# the modifications in the patch above.\n\n\n# Reset t5304-prune.sh back to the original, then add an exit statement\n# _after_ test 4. The test will fail. Then cd into 'trash directory.t5304-prune'\n# and again 'git prune --expire 2.weeks.ago' will succeed.\n$ rm -rf 'trash directory.t5304-prune'\n$ ./t5304-prune.sh\n$ cd 'trash directory.t5304-prune'\n$ git prune --expire 2.weeks.ago\n# success, correct file times are printed out.\n\n# Now do the same thing as above, but this time run 'git gc' before running\n# prune.\n$ rm -rf 'trash directory.t5304-prune'\n$ ./t5304-prune.sh\n$ cd 'trash directory.t5304-prune'\n$ git gc\n# fails to remove appropriate file, all file times are from atime\n$ git prune --expire 2.weeks.ago\n# fails to remove the appropriate file, all file times are from atime\n\nAfter some amount of time (minutes), git gc succeeds in removing the\nobject and prints out the correct time values.\n\n-brandon\n"},{"id":"105172","messageId":"UeTsTvDfFT9ToLdKosYcgFRx4HzMKUFpn67jts4q-qNPzYZVyG8DQQ@cipher.nrlssc.navy.mil","threadId":"17758","inReplyTo":"3a69fa7c0902170641m2ba9b8aeo950c9e3868ba6244@mail.gmail.com","subject":"Re: not all tests pass when on an NFS partition","fromName":"Brandon Casey","fromEmail":"casey@nrlssc.navy.mil","sentAt":"2009-02-17T16:32:23Z","receivedAt":"2009-02-17T16:32:23Z","isPatch":false,"sender":{"key":"drafnel@gmail.com","avatar":"https://avatars.githubusercontent.com/u/921167?v=4"},"body":"E R wrote:\n> Btw - what kind of risks are involved in using git when test\n> t5304-prune.sh doesn't pass? Do we risk repository corruption?\n\nFor the case in the test, git-prune is failing to remove an object\nthat it should, which won't cause repo corruption. It will just\ncause retention of an object for longer than it is needed. I suspect\nthis is caused by some flaw in the NFS client (disclosure: IANAKH).\n\nIn practice, I don't think you'll experience this flaw. I think it\nshows up in the test because we are manually adjusting the mtime of\nthe objects and then immediately trying to prune them and this must\ntweak whatever code has a bug.\n\nI have been using git on 5.2 for a couple of months now with no problem.\nAdmittedly, prune is a rare operation, usually triggered by git-gc.\n\n-brandon\n"}]}