{"thread":{"id":"27036","subject":"t3306 failure with v1.7.5-rc1","startedAt":"2011-04-08T07:06:46Z","lastAt":"2011-04-08T18:53:25Z","messageCount":7,"participants":["Michael J Gruber","Johannes Sixt","Jeff King","Junio C Hamano"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"165404","messageId":"4D9EB406.5080302@viscovery.net","threadId":"27036","inReplyTo":"4D9ECF7C.6010709@drmicha.warpmail.net","subject":"Re: t3306 failure with v1.7.5-rc1","fromName":"Johannes Sixt","fromEmail":"j.sixt@viscovery.net","sentAt":"2011-04-08T07:06:46Z","receivedAt":"2011-04-08T07:06:46Z","isPatch":false,"sender":{"key":"j6t@kdbg.org","avatar":"https://avatars.githubusercontent.com/u/14810926?v=4"},"body":"Am 4/8/2011 11:03, schrieb Michael J Gruber:\n> I get this stupid test failure in test 3 of t3306. The problem is that a\n> dangling commit does not get pruned away when it should:\n> \n> 3rd\n> test_must_fail: command succeeded: git cat-file -p\n> 5ee1c35e83ea47cd3cc4f8cbee0568915fbbbd29\n> not ok - 4 verify that commits are gone\n> \n> It's a system where make complains about funny clock (I dunno why) but\n> can we make this more robust? The following helps with \"sleep 5\" but not\n> with \"sleep 4\". test_tick does not help. What's going on?\n\nLooking at the time of day of your last emails (it's ~2 hours in the\nfuture), I'd say something is fishy with your system's clock. Fix that first.\n\n-- Hannes\n"},{"id":"165406","messageId":"4D9EB57D.1060402@drmicha.warpmail.net","threadId":"27036","inReplyTo":"4D9EB406.5080302@viscovery.net","subject":"Re: t3306 failure with v1.7.5-rc1","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2011-04-08T07:13:01Z","receivedAt":"2011-04-08T07:13:01Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Johannes Sixt venit, vidit, dixit 08.04.2011 09:06:\n> Am 4/8/2011 11:03, schrieb Michael J Gruber:\n>> I get this stupid test failure in test 3 of t3306. The problem is that a\n>> dangling commit does not get pruned away when it should:\n>>\n>> 3rd\n>> test_must_fail: command succeeded: git cat-file -p\n>> 5ee1c35e83ea47cd3cc4f8cbee0568915fbbbd29\n>> not ok - 4 verify that commits are gone\n>>\n>> It's a system where make complains about funny clock (I dunno why) but\n>> can we make this more robust? The following helps with \"sleep 5\" but not\n>> with \"sleep 4\". test_tick does not help. What's going on?\n> \n> Looking at the time of day of your last emails (it's ~2 hours in the\n> future), I'd say something is fishy with your system's clock. Fix that first.\n\nWell, while you're right about that, an offset should not matter for the\ntest, right?\n\nAbout my clock: The result of booting a \"different\" system on that\nbox... (In this case even a kubuntu using hw clock in tz...). thanks for\npointing it out!\n\nIn any case, that test was on a different box with clean checkout and\nmake clean. I don't think anyone jumped the clock right in between each\nof my 20 or so test runs.\n\nMichael\n"},{"id":"165408","messageId":"4D9EBC41.4080501@drmicha.warpmail.net","threadId":"27036","inReplyTo":"4D9EB57D.1060402@drmicha.warpmail.net","subject":"\"prune\" prone to clock skew (Re: t3306 failure with v1.7.5-rc1)","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2011-04-08T07:41:53Z","receivedAt":"2011-04-08T07:41:53Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Michael J Gruber venit, vidit, dixit 08.04.2011 09:13:\n> Johannes Sixt venit, vidit, dixit 08.04.2011 09:06:\n>> Am 4/8/2011 11:03, schrieb Michael J Gruber:\n>>> I get this stupid test failure in test 3 of t3306. The problem is that a\n>>> dangling commit does not get pruned away when it should:\n>>>\n>>> 3rd\n>>> test_must_fail: command succeeded: git cat-file -p\n>>> 5ee1c35e83ea47cd3cc4f8cbee0568915fbbbd29\n>>> not ok - 4 verify that commits are gone\n>>>\n>>> It's a system where make complains about funny clock (I dunno why) but\n\n(I know now.)\n\n>>> can we make this more robust? The following helps with \"sleep 5\" but not\n>>> with \"sleep 4\". test_tick does not help. What's going on?\n\nAdding one more observation from that system:\n\nrm GIT-VERSION-FILE ;date; sh ./GIT-VERSION-GEN; date; ls -l --full-time\nGIT-VERSION-FILE\n\nFri Apr  8 09:27:40 CEST 2011\nGIT_VERSION = 1.7.5.rc1\nFri Apr  8 09:27:41 CEST 2011\n-rw-r----- 1 mjg mjg 24 2011-04-08 09:27:47.000000000 +0200 GIT-VERSION-FILE\n\nNote that the second \"date\" happens after the creation of\nGIT-VERSION-FILE. I can even do this with a simple\n\nrm a;date; touch a; date; ls -l --full-time a\nFri Apr  8 09:31:04 CEST 2011\nFri Apr  8 09:31:04 CEST 2011\n-rw-r----- 1 mjg mjg 0 2011-04-08 09:31:11.000000000 +0200 a\n\nSuffice it to say that this is on NFS, and of course the creation date\ncomes from the NFS server, the \"date\" from the client... OK, I got that.\nScary.\n\nEXPLANATION:\n\nI guess \"prune\" looks at some file time stamps (rather than parsing the\ncommit time proper) so that it is prone to the same client/server clock\nskew problems when the repo is not on the local file system.\n\nPROBLEM (?):\n\nI really hope this does not go both ways - imagine a messed server\nturning it's clock a few weeks back by accident, and running \"git prune\"\non the client, or a messed client turning the clock forward... Do we do\nproper checks before removing something?\n\nI mean, there's a difference between missing that something is stale and\nmissing that it is not...\n\nMichael\n"},{"id":"165403","messageId":"4D9ECF7C.6010709@drmicha.warpmail.net","threadId":"27036","inReplyTo":null,"subject":"t3306 failure with v1.7.5-rc1","fromName":"Michael J Gruber","fromEmail":"git@drmicha.warpmail.net","sentAt":"2011-04-08T09:03:56Z","receivedAt":"2011-04-08T09:03:56Z","isPatch":false,"sender":{"key":"git@grubix.eu","avatar":"https://avatars.githubusercontent.com/u/233215?v=4"},"body":"Heya,\n\nI get this stupid test failure in test 3 of t3306. The problem is that a\ndangling commit does not get pruned away when it should:\n\n3rd\ntest_must_fail: command succeeded: git cat-file -p\n5ee1c35e83ea47cd3cc4f8cbee0568915fbbbd29\nnot ok - 4 verify that commits are gone\n\nIt's a system where make complains about funny clock (I dunno why) but\ncan we make this more robust? The following helps with \"sleep 5\" but not\nwith \"sleep 4\". test_tick does not help. What's going on?\n\n---\n\ndiff --git i/t/t3306-notes-prune.sh w/t/t3306-notes-prune.sh\nindex c428217..da76463 100755\n--- i/t/t3306-notes-prune.sh\n+++ w/t/t3306-notes-prune.sh\n@@ -62,7 +62,9 @@ test_expect_success 'remove some commits' '\n\n \tgit reset --hard HEAD~1 &&\n \tgit reflog expire --expire=now HEAD &&\n+\tsleep 5 &&\n \tgit gc --prune=now\n '\n\n test_expect_success 'verify that commits are gone' '\n"},{"id":"165428","messageId":"20110408163021.GB32709@sigill.intra.peff.net","threadId":"27036","inReplyTo":"4D9EBC41.4080501@drmicha.warpmail.net","subject":"Re: \"prune\" prone to clock skew (Re: t3306 failure with v1.7.5-rc1)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-04-08T16:30:22Z","receivedAt":"2011-04-08T16:30:22Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Apr 08, 2011 at 09:41:53AM +0200, Michael J Gruber wrote:\n\n> EXPLANATION:\n> \n> I guess \"prune\" looks at some file time stamps (rather than parsing the\n> commit time proper) so that it is prone to the same client/server clock\n> skew problems when the repo is not on the local file system.\n\nOf course. Remember that prune is removing all sorts of objects, not\njust commits. In the case of a tree or blob, we have nothing else to go\non besides the filesystem timestamp.\n\n> PROBLEM (?):\n> \n> I really hope this does not go both ways - imagine a messed server\n> turning it's clock a few weeks back by accident, and running \"git prune\"\n> on the client, or a messed client turning the clock forward... Do we do\n> proper checks before removing something?\n\nI don't think anybody has ever cared. These things are, after all,\nunreferenced by either current refs or the reflog. I think the purpose\nof the 2-week grace period is more to prevent prune from deleting\nobjects from an in-progress operation before that operation has had a\nchance to write its refs.\n\nSo one problem scenario is that the prune on the machine with the bad\nclock would delete objects from something like a push in progress.\n\nBut I think it's just so unlikely to happen that your repo is on a\nremote server, _and_ the client and server clocks are skewed by more\nthan two weeks, _and_ you're running prune, _and_ somebody is pushing\ninto the repo simultaneously, that nobody has really bothered to try to\ndo better.\n\nNote that if you access the same repo over a filesystem share by two\nclients with different clocks, you can also expire reflog entries\nprematurely. But that is a 90-day window.\n\nSo I don't know that it's worth fixing, nor do I know what your \"proper\nchecks\" would be. We could open up commits we're about to prune and\ncheck their timestamps for age. That wouldn't help blobs and trees. For\nthem, we could make a guess as to their age by seeing if they're\nconnected to any dangling commits, and using the commit timestamp. But\nthat will only cover some objects, and it is also still vulnerable to\ncertain types of clock skew (though it is more robust, since you would\nneed a client and server with one type writing the objects, and then\nanother skewed client doing the prune, which is even less likely).\n\n-Peff\n"},{"id":"165436","messageId":"7vei5c4lwh.fsf@alter.siamese.dyndns.org","threadId":"27036","inReplyTo":"20110408163021.GB32709@sigill.intra.peff.net","subject":"Re: \"prune\" prone to clock skew (Re: t3306 failure with v1.7.5-rc1)","fromName":"Junio C Hamano","fromEmail":"gitster@pobox.com","sentAt":"2011-04-08T18:51:10Z","receivedAt":"2011-04-08T18:51:10Z","isPatch":false,"sender":{"key":"gitster@pobox.com","avatar":"https://avatars.githubusercontent.com/u/54884?v=4"},"body":"Jeff King <peff@peff.net> writes:\n\n> So I don't know that it's worth fixing, nor do I know what your \"proper\n> checks\" would be. We could open up commits we're about to prune and\n> check their timestamps for age. That wouldn't help blobs and trees. For\n> them, we could make a guess as to their age by seeing if they're\n> connected to any dangling commits, and using the commit timestamp. But\n> that will only cover some objects, and it is also still vulnerable to\n> certain types of clock skew (though it is more robust, since you would\n> need a client and server with one type writing the objects, and then\n> another skewed client doing the prune, which is even less likely).\n\nI don't personally think this is worth addressing, but as I was Cc'ed,\nI'll try.\n\nIf we at least know that the object store is contained within a single\nfilesystem whose clock is at least guaranteed to monotonically increase,\none possible workaround would be to touch a garbage file and read its\ntimestamp, instead of grabbing the current time from the local clock, no?\n"},{"id":"165437","messageId":"20110408185325.GA6337@sigill.intra.peff.net","threadId":"27036","inReplyTo":"7vei5c4lwh.fsf@alter.siamese.dyndns.org","subject":"Re: \"prune\" prone to clock skew (Re: t3306 failure with v1.7.5-rc1)","fromName":"Jeff King","fromEmail":"peff@peff.net","sentAt":"2011-04-08T18:53:25Z","receivedAt":"2011-04-08T18:53:25Z","isPatch":false,"sender":{"key":"peff@peff.net","avatar":"https://avatars.githubusercontent.com/u/45925?v=4"},"body":"On Fri, Apr 08, 2011 at 11:51:10AM -0700, Junio C Hamano wrote:\n\n> I don't personally think this is worth addressing, but as I was Cc'ed,\n> I'll try.\n> \n> If we at least know that the object store is contained within a single\n> filesystem whose clock is at least guaranteed to monotonically increase,\n> one possible workaround would be to touch a garbage file and read its\n> timestamp, instead of grabbing the current time from the local clock, no?\n\nYeah, that is a much more accurate and simpler fix. Like you, I don't\nreally think it's worth worrying about, but if somebody wanted to, that\nis much better than the fix I mentioned.\n\n-Peff\n"}]}