{"thread":{"id":"20225","subject":"[JGIT] Re: blinking test WindowCacheGetTest.testCache_TooSmallLimit","startedAt":"2009-07-24T22:51:18Z","lastAt":"2009-07-25T23:30:26Z","messageCount":6,"participants":["Shawn O. Pearce","Robin Rosenberg"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"118680","messageId":"20090724225118.GZ11191@spearce.org","threadId":"20225","inReplyTo":"85647ef50907220623i2b7e50dal67650a638921ec0f@mail.gmail.com","subject":"[JGIT] Re: blinking test WindowCacheGetTest.testCache_TooSmallLimit","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-07-24T22:51:18Z","receivedAt":"2009-07-24T22:51:18Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Constantine Plotnikov wrote:\n> The test WindowCacheGetTest.testCache_TooSmallLimit sometimes fails\n> (on less than third of runs on Windows) with the following stacktrace:\n> \n> junit.framework.AssertionFailedError\n> at junit.framework.Assert.fail(Assert.java:47)\n> at junit.framework.Assert.assertTrue(Assert.java:20)\n> at junit.framework.Assert.assertTrue(Assert.java:27)\n> at org.spearce.jgit.lib.WindowCacheGetTest.checkLimits(WindowCacheGetTest.java:112)\n> at org.spearce.jgit.lib.WindowCacheGetTest.testCache_TooSmallLimit(WindowCacheGetTest.java:106)\n> \n> This happens on Windows and Linux, but I do not know about\n> frequency on Linux.\n\nI'd say the frequency on Linux is about 1/6 for me. I have yet to\nbe bothered enough to track it down, but its starting to get there.\nMaybe I'll try to look at it tomorrow.\n\n-- \nShawn.\n"},{"id":"118734","messageId":"200907251934.37145.robin.rosenberg.lists@dewire.com","threadId":"20225","inReplyTo":"20090724225118.GZ11191@spearce.org","subject":"Re: [JGIT] Re: blinking test WindowCacheGetTest.testCache_TooSmallLimit","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg.lists@dewire.com","sentAt":"2009-07-25T17:34:36Z","receivedAt":"2009-07-25T17:34:36Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"lördag 25 juli 2009 00:51:18 skrev \"Shawn O. Pearce\" :\n> Constantine Plotnikov wrote:\n> > The test WindowCacheGetTest.testCache_TooSmallLimit sometimes fails\n> > (on less than third of runs on Windows) with the following stacktrace:\n> > \n> > junit.framework.AssertionFailedError\n> > at junit.framework.Assert.fail(Assert.java:47)\n> > at junit.framework.Assert.assertTrue(Assert.java:20)\n> > at junit.framework.Assert.assertTrue(Assert.java:27)\n> > at org.spearce.jgit.lib.WindowCacheGetTest.checkLimits(WindowCacheGetTest.java:112)\n> > at org.spearce.jgit.lib.WindowCacheGetTest.testCache_TooSmallLimit(WindowCacheGetTest.java:106)\n> > \n> > This happens on Windows and Linux, but I do not know about\n> > frequency on Linux.\n> \n> I'd say the frequency on Linux is about 1/6 for me. I have yet to\n> be bothered enough to track it down, but its starting to get there.\n> Maybe I'll try to look at it tomorrow.\n\nCould it be threading-related (cache). I've never seen it on the machine where I build for\nthe update site and it is a single core machine, but I \"this\" machine that has two cores it happens, not\nas offen as 1/6, but enough to annoy me a bit, perhaps 1/20. If it is related to bad synchronization\nit should happen more often the more cores you have.\n\n-- robin\n"},{"id":"118771","messageId":"20090725194254.GC11191@spearce.org","threadId":"20225","inReplyTo":"200907251934.37145.robin.rosenberg.lists@dewire.com","subject":"Re: [JGIT] Re: blinking test WindowCacheGetTest.testCache_TooSmallLimit","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-07-25T19:42:54Z","receivedAt":"2009-07-25T19:42:54Z","isPatch":false,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Robin Rosenberg wrote:\n> > Constantine Plotnikov wrote:\n> > > The test WindowCacheGetTest.testCache_TooSmallLimit sometimes fails\n> > > (on less than third of runs on Windows) with the following stacktrace:\n> \n> Could it be threading-related (cache). I've never seen it on the machine where I build for\n> the update site and it is a single core machine, but I \"this\" machine that has two cores it happens, not\n> as offen as 1/6, but enough to annoy me a bit, perhaps 1/20. If it is related to bad synchronization\n> it should happen more often the more cores you have.\n\nNo, its not threading related. I think I at least partially\nunderstand what is going on.\n\nWindowCache's base class, OffsetCache, does evictions by randomly\nselecting a starting point in the table, and then scanning at most\nevictBatch segments of the table before giving up.\n\nWhen testCache_TooSmallLimit needs to access another window it must\nfirst evict the current window, as the cache size is set to exactly\n1 window in this test.\n\nIf the RNG selects the starting point at an empty segment, and\nevictBatch is smaller than the total table size, we may not scan\nthe segment which contains an open window. The eviction stops\nbefore it actually expires an entry, and the cache is permitted to\nallocate more than it was supposed to.\n\nThe test failure is random due to two different things going on:\n\n - The starting point for eviction is determined by an RNG, whose\n seed is initialized randomly at the start of the tests.\n\n - The position of a given entry in the segment table is partially\n determined by the System.identityHashCode of the PackFile instance\n it is being loaded for. Since that differs as the JVM allocs\n and reclaims objects, there is some randomness caused by the\n order tests are run in.\n\nHowever, in this test I'm still not quite sure why its failing,\nbecause evictBatch is equal to the table size (both 2), so any\neviction here should have scanned the full table.\n\n-- \nShawn.\n"},{"id":"118773","messageId":"20090725200056.GD11191@spearce.org","threadId":"20225","inReplyTo":"20090725194254.GC11191@spearce.org","subject":"[JGIT PATCH] Fix WindowCacheGetTest.testCache_TooSmallLimit failures","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-07-25T20:00:56Z","receivedAt":"2009-07-25T20:00:56Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Ever since 2d77d30b5f when I rewrote WindowCache we have been seeing\nrandom failures inside of the TooSmallLimit test case.\n\nThese test failures have been occurring because the cache contained\nmore open bytes than it was configured to permit.\n\nThe cache was permitted to open more bytes than its configured limit\nbecause the eviction routine was always skipping the last bucket\nunder some conditions. If the cache table was sized the same as its\nevictBatch, which happens for any fairly small table, the eviction\nroutine broke too early if it started at a non-zero position in the\ntable and wrapped around during its search. By breaking too early\nthe routine did not actually perform an eviction, leaving windows\nopen it should have closed.\n\nSigned-off-by: Shawn O. Pearce \n---\n > Robin Rosenberg wrote:\n > > > Constantine Plotnikov wrote:\n > > > > The test WindowCacheGetTest.testCache_TooSmallLimit sometimes fails\n > > > > (on less than third of runs on Windows) with the following stacktrace:\n > > \n > > Could it be threading-related (cache). I've never seen it on the machine where I build for\n > > the update site and it is a single core machine, but I \"this\" machine that has two cores it happens, not\n > > as offen as 1/6, but enough to annoy me a bit, perhaps 1/20. If it is related to bad synchronization\n > > it should happen more often the more cores you have.\n \n I think this fixes it.\n\n .../src/org/spearce/jgit/lib/OffsetCache.java | 7 ++-----\n 1 files changed, 2 insertions(+), 5 deletions(-)\n\ndiff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/OffsetCache.java b/org.spearce.jgit/src/org/spearce/jgit/lib/OffsetCache.java\nindex b81c7e0..7ac532d 100644\n--- a/org.spearce.jgit/src/org/spearce/jgit/lib/OffsetCache.java\n+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/OffsetCache.java\n@@ -253,12 +253,11 @@ private void hit(final Ref r) {\n \t}\n \n \tprivate void evict() {\n-\t\tfinal int start = rng.nextInt(tableSize);\n-\t\tint ptr = start;\n \t\twhile (isFull()) {\n+\t\t\tint ptr = rng.nextInt(tableSize);\n \t\t\tEntry old = null;\n \t\t\tint slot = 0;\n-\t\t\tfor (int b = evictBatch - 1; b >= 0; b--) {\n+\t\t\tfor (int b = evictBatch - 1; b >= 0; b--, ptr++) {\n \t\t\t\tif (tableSize <= ptr)\n \t\t\t\t\tptr = 0;\n \t\t\t\tfor (Entry e = table.get(ptr); e != null; e = e.next) {\n@@ -269,8 +268,6 @@ private void evict() {\n \t\t\t\t\t\tslot = ptr;\n \t\t\t\t\t}\n \t\t\t\t}\n-\t\t\t\tif (++ptr == start)\n-\t\t\t\t\treturn;\n \t\t\t}\n \t\t\tif (old != null) {\n \t\t\t\told.kill();\n-- \n1.6.4.rc2.216.g769fa\n"},{"id":"118806","messageId":"200907260125.32306.robin.rosenberg.lists@dewire.com","threadId":"20225","inReplyTo":"20090725200056.GD11191@spearce.org","subject":"Re: [JGIT PATCH] Fix WindowCacheGetTest.testCache_TooSmallLimit failures","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg.lists@dewire.com","sentAt":"2009-07-25T23:25:31Z","receivedAt":"2009-07-25T23:25:31Z","isPatch":true,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"lördag 25 juli 2009 22:00:56 skrev \"Shawn O. Pearce\" :\n> Ever since 2d77d30b5f when I rewrote WindowCache we have been seeing\n> random failures inside of the TooSmallLimit test case.\n> \n> These test failures have been occurring because the cache contained\n> more open bytes than it was configured to permit.\n> \n> The cache was permitted to open more bytes than its configured limit\n> because the eviction routine was always skipping the last bucket\n> under some conditions. If the cache table was sized the same as its\n> evictBatch, which happens for any fairly small table, the eviction\n> routine broke too early if it started at a non-zero position in the\n> table and wrapped around during its search. By breaking too early\n> the routine did not actually perform an eviction, leaving windows\n> open it should have closed.\n\nWe should have a test for that then.\n\n-- robin\n"},{"id":"118807","messageId":"20090725233026.GH11191@spearce.org","threadId":"20225","inReplyTo":"200907260125.32306.robin.rosenberg.lists@dewire.com","subject":"Re: [JGIT PATCH] Fix WindowCacheGetTest.testCache_TooSmallLimit failures","fromName":"Shawn O. Pearce","fromEmail":"spearce@spearce.org","sentAt":"2009-07-25T23:30:26Z","receivedAt":"2009-07-25T23:30:26Z","isPatch":true,"sender":{"key":"spearce@spearce.org","avatar":"https://avatars.githubusercontent.com/u/34844?v=4"},"body":"Robin Rosenberg wrote:\n> l?rdag 25 juli 2009 22:00:56 skrev \"Shawn O. Pearce\" :\n> > Ever since 2d77d30b5f when I rewrote WindowCache we have been seeing\n> > random failures inside of the TooSmallLimit test case.\n> > \n> > These test failures have been occurring because the cache contained\n> > more open bytes than it was configured to permit.\n> > \n> > The cache was permitted to open more bytes than its configured limit\n> > because the eviction routine was always skipping the last bucket\n> > under some conditions. If the cache table was sized the same as its\n> > evictBatch, which happens for any fairly small table, the eviction\n> > routine broke too early if it started at a non-zero position in the\n> > table and wrapped around during its search. By breaking too early\n> > the routine did not actually perform an eviction, leaving windows\n> > open it should have closed.\n> \n> We should have a test for that then.\n\nWe did, this one. :-)\n\n-- \nShawn.\n"}]}