{"thread":{"id":"19675","subject":"[BUG?] Working tree getting out of date \"spontaneously\"","startedAt":"2009-06-05T12:24:44Z","lastAt":"2009-06-05T21:38:24Z","messageCount":13,"participants":["Björn Steinbrink","Robin Rosenberg","Theodore Tso","Linus Torvalds"],"isPatch":false,"patchVersion":null,"patchTotal":null},"messages":[{"id":"115501","messageId":"20090605122444.GA11035@atjola.homenet","threadId":"19675","inReplyTo":null,"subject":"[BUG?] Working tree getting out of date \"spontaneously\"","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-06-05T12:24:44Z","receivedAt":"2009-06-05T12:24:44Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"Hi,\n\ncan't make any sense out of this at all:\n\ndoener@atjola:git (master) $ git reset --hard HEAD@{1}\nHEAD is now at b11cf09 Merge branch 'da/pretty-tempname'\ndoener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD\n\ndoener@atjola:git (master) $ git pull\nFrom git://git.kernel.org/pub/scm/git/git\n b11cf09..6096d75 master -> origin/master\nUpdating b11cf09..6096d75\nFast forward\n Documentation/RelNotes-1.6.3.2.txt | 12 +++++-------\n Documentation/git.txt | 7 ++++++-\n contrib/completion/git-completion.bash | 12 ++++++++++--\n grep.c | 6 +++++-\n 4 files changed, 26 insertions(+), 11 deletions(-)\n\ndoener@atjola:git (master) $ git diff-index --name-only HEAD\ndoener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n\n*wait a minute, doing nothing*\n\ndoener@atjola:git (master) $ git diff-index --name-only HEAD\nDocumentation/RelNotes-1.6.3.2.txt\nDocumentation/git.txt\ncontrib/completion/git-completion.bash\ngrep.c\n\ndoener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n\ndoener@atjola:git (master) $ git diff-index --stat HEAD\n 0 files changed, 0 insertions(+), 0 deletions(-)\n\ndoener@atjola:git (master) $ git diff-index --name-only HEAD\nDocumentation/RelNotes-1.6.3.2.txt\nDocumentation/git.txt\ncontrib/completion/git-completion.bash\ngrep.c\n\n\nRunning \"git status\" seems to fix things.\n\nBjörn, confused\n"},{"id":"115502","messageId":"200906051514.15876.robin.rosenberg.lists@dewire.com","threadId":"19675","inReplyTo":"20090605122444.GA11035@atjola.homenet","subject":"Re: [BUG?] Working tree getting out of date \"spontaneously\"","fromName":"Robin Rosenberg","fromEmail":"robin.rosenberg.lists@dewire.com","sentAt":"2009-06-05T13:14:15Z","receivedAt":"2009-06-05T13:14:15Z","isPatch":false,"sender":{"key":"robin.rosenberg@dewire.com","avatar":"https://avatars.githubusercontent.com/u/46357?v=4"},"body":"fredag 05 juni 2009 14:24:44 skrev Björn Steinbrink:\n> Hi,\n> \n> can't make any sense out of this at all:\n> \n> doener@atjola:git (master) $ git reset --hard HEAD@{1}\n> HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'\n> doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD\n> \n> doener@atjola:git (master) $ git pull\n> From git://git.kernel.org/pub/scm/git/git\n> b11cf09..6096d75 master -> origin/master\n> Updating b11cf09..6096d75\n> Fast forward\n> Documentation/RelNotes-1.6.3.2.txt | 12 +++++-------\n> Documentation/git.txt | 7 ++++++-\n> contrib/completion/git-completion.bash | 12 ++++++++++--\n> grep.c | 6 +++++-\n> 4 files changed, 26 insertions(+), 11 deletions(-)\n> \n> doener@atjola:git (master) $ git diff-index --name-only HEAD\n> doener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n> \n> *wait a minute, doing nothing*\n> \n> doener@atjola:git (master) $ git diff-index --name-only HEAD\n> Documentation/RelNotes-1.6.3.2.txt\n> Documentation/git.txt\n> contrib/completion/git-completion.bash\n> grep.c\n> \n> doener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n> \n> doener@atjola:git (master) $ git diff-index --stat HEAD\n> 0 files changed, 0 insertions(+), 0 deletions(-)\n> \n> doener@atjola:git (master) $ git diff-index --name-only HEAD\n> Documentation/RelNotes-1.6.3.2.txt\n> Documentation/git.txt\n> contrib/completion/git-completion.bash\n> grep.c\n> \n> \n> Running \"git status\" seems to fix things.\n> \n> Björn, confused\n\nWhat file system and OS?\n\n-- robin\n"},{"id":"115503","messageId":"20090605132126.GB11035@atjola.homenet","threadId":"19675","inReplyTo":"20090605122444.GA11035@atjola.homenet","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-06-05T13:21:26Z","receivedAt":"2009-06-05T13:21:26Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.06.05 14:24:44 +0200, Björn Steinbrink wrote:\n> Hi,\n> \n> can't make any sense out of this at all:\n> \n> doener@atjola:git (master) $ git reset --hard HEAD@{1}\n> HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'\n> doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD\n> \n> doener@atjola:git (master) $ git pull\n> >From git://git.kernel.org/pub/scm/git/git\n> b11cf09..6096d75 master -> origin/master\n> Updating b11cf09..6096d75\n> Fast forward\n> Documentation/RelNotes-1.6.3.2.txt | 12 +++++-------\n> Documentation/git.txt | 7 ++++++-\n> contrib/completion/git-completion.bash | 12 ++++++++++--\n> grep.c | 6 +++++-\n> 4 files changed, 26 insertions(+), 11 deletions(-)\n> \n> doener@atjola:git (master) $ git diff-index --name-only HEAD\n> doener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n> \n> *wait a minute, doing nothing*\n> \n> doener@atjola:git (master) $ git diff-index --name-only HEAD\n> Documentation/RelNotes-1.6.3.2.txt\n> Documentation/git.txt\n> contrib/completion/git-completion.bash\n> grep.c\n\nHm, looks like this is not a git bug. Went back to 1.5.4, and even that\nshows the error. So I actually looked at the files, and indeed, the file\nin the working tree gets modified. stat(1) shows:\n\nRight after the merge:\n File: `grep.c'\n Size: 16274 Blocks: 32 IO Block: 4096 regular file\nDevice: fd03h/64771d Inode: 5933481 Links: 1\nAccess: (0644/-rw-r--r--) Uid: ( 1000/ doener) Gid: ( 1000/ doener)\nAccess: 2009-06-05 15:02:14.000000000 +0200\nModify: 2009-06-05 15:02:14.000000000 +0200\nChange: 2009-06-05 15:02:14.000000000 +0200\n\n60 seconds later:\n File: `grep.c'\n Size: 16274 Blocks: 32 IO Block: 4096 regular file\nDevice: fd03h/64771d Inode: 5933481 Links: 1\nAccess: (0644/-rw-r--r--) Uid: ( 1000/ doener) Gid: ( 1000/ doener)\nAccess: 2009-06-05 15:02:14.000000000 +0200\nModify: 2009-06-05 15:02:14.000000000 +0200\nChange: 2009-06-05 15:02:48.000000000 +0200\n\nSo the ctime got modified. I don't have any fancy indexing stuff\nrunning, and inotify doesn't see any events either while the ctime is\nchanged.\n\nThe only thing I changed lately was upgrading to 2.6.30-rc8 and going\nfrom ext3 to ext4. As the ctime change always seems to happen around 30\nseconds after the real change, I kind of suspect ext4 to be guilty.\nTed, is that possible?\n\nFS is mounted as:\n/dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)\n\n\nBjörn\n"},{"id":"115504","messageId":"20090605132331.GC11035@atjola.homenet","threadId":"19675","inReplyTo":"200906051514.15876.robin.rosenberg.lists@dewire.com","subject":"Re: [BUG?] Working tree getting out of date \"spontaneously\"","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-06-05T13:23:31Z","receivedAt":"2009-06-05T13:23:31Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.06.05 15:14:15 +0200, Robin Rosenberg wrote:\n> fredag 05 juni 2009 14:24:44 skrev Björn Steinbrink:\n> > Hi,\n> > \n> > can't make any sense out of this at all:\n> > \n> > doener@atjola:git (master) $ git reset --hard HEAD@{1}\n> > HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'\n> > doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD\n> > \n> > doener@atjola:git (master) $ git pull\n> > From git://git.kernel.org/pub/scm/git/git\n> > b11cf09..6096d75 master -> origin/master\n> > Updating b11cf09..6096d75\n> > Fast forward\n> > Documentation/RelNotes-1.6.3.2.txt | 12 +++++-------\n> > Documentation/git.txt | 7 ++++++-\n> > contrib/completion/git-completion.bash | 12 ++++++++++--\n> > grep.c | 6 +++++-\n> > 4 files changed, 26 insertions(+), 11 deletions(-)\n> > \n> > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > doener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n> > \n> > *wait a minute, doing nothing*\n> > \n> > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > Documentation/RelNotes-1.6.3.2.txt\n> > Documentation/git.txt\n> > contrib/completion/git-completion.bash\n> > grep.c\n> > \n> > doener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n> > \n> > doener@atjola:git (master) $ git diff-index --stat HEAD\n> > 0 files changed, 0 insertions(+), 0 deletions(-)\n> > \n> > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > Documentation/RelNotes-1.6.3.2.txt\n> > Documentation/git.txt\n> > contrib/completion/git-completion.bash\n> > grep.c\n> > \n> > \n> > Running \"git status\" seems to fix things.\n> > \n> > Björn, confused\n> \n> What file system and OS?\n\nLinux 2.6.30-rc8, ext4. ctime gets updated for no apparent reason, see\nthe other mail I just sent.\n\nThanks,\nBjörn\n"},{"id":"115506","messageId":"20090605141209.GA764@atjola.homenet","threadId":"19675","inReplyTo":"20090605132126.GB11035@atjola.homenet","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-06-05T14:12:09Z","receivedAt":"2009-06-05T14:12:09Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.06.05 15:21:26 +0200, Björn Steinbrink wrote:\n> On 2009.06.05 14:24:44 +0200, Björn Steinbrink wrote:\n> > Hi,\n> > \n> > can't make any sense out of this at all:\n> > \n> > doener@atjola:git (master) $ git reset --hard HEAD@{1}\n> > HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'\n> > doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD\n> > \n> > doener@atjola:git (master) $ git pull\n> > >From git://git.kernel.org/pub/scm/git/git\n> > b11cf09..6096d75 master -> origin/master\n> > Updating b11cf09..6096d75\n> > Fast forward\n> > Documentation/RelNotes-1.6.3.2.txt | 12 +++++-------\n> > Documentation/git.txt | 7 ++++++-\n> > contrib/completion/git-completion.bash | 12 ++++++++++--\n> > grep.c | 6 +++++-\n> > 4 files changed, 26 insertions(+), 11 deletions(-)\n> > \n> > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > doener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n> > \n> > *wait a minute, doing nothing*\n> > \n> > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > Documentation/RelNotes-1.6.3.2.txt\n> > Documentation/git.txt\n> > contrib/completion/git-completion.bash\n> > grep.c\n> \n> Hm, looks like this is not a git bug. Went back to 1.5.4, and even that\n> shows the error. So I actually looked at the files, and indeed, the file\n> in the working tree gets modified. stat(1) shows:\n> \n> Right after the merge:\n> File: `grep.c'\n> Size: 16274 Blocks: 32 IO Block: 4096 regular file\n> Device: fd03h/64771d Inode: 5933481 Links: 1\n> Access: (0644/-rw-r--r--) Uid: ( 1000/ doener) Gid: ( 1000/ doener)\n> Access: 2009-06-05 15:02:14.000000000 +0200\n> Modify: 2009-06-05 15:02:14.000000000 +0200\n> Change: 2009-06-05 15:02:14.000000000 +0200\n> \n> 60 seconds later:\n> File: `grep.c'\n> Size: 16274 Blocks: 32 IO Block: 4096 regular file\n> Device: fd03h/64771d Inode: 5933481 Links: 1\n> Access: (0644/-rw-r--r--) Uid: ( 1000/ doener) Gid: ( 1000/ doener)\n> Access: 2009-06-05 15:02:14.000000000 +0200\n> Modify: 2009-06-05 15:02:14.000000000 +0200\n> Change: 2009-06-05 15:02:48.000000000 +0200\n> \n> So the ctime got modified. I don't have any fancy indexing stuff\n> running, and inotify doesn't see any events either while the ctime is\n> changed.\n> \n> The only thing I changed lately was upgrading to 2.6.30-rc8 and going\n> from ext3 to ext4. As the ctime change always seems to happen around 30\n> seconds after the real change, I kind of suspect ext4 to be guilty.\n> Ted, is that possible?\n> \n> FS is mounted as:\n> /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)\n\nHm, yup, seems to be ext4 related, doesn't happen on tmpfs.\n\nBjörn\n"},{"id":"115507","messageId":"20090605144748.GB764@atjola.homenet","threadId":"19675","inReplyTo":"20090605141209.GA764@atjola.homenet","subject":"Re: [BUG? ext4] git working tree getting out of date \"spontaneously\"","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-06-05T14:47:48Z","receivedAt":"2009-06-05T14:47:48Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.06.05 16:12:09 +0200, Björn Steinbrink wrote:\n> On 2009.06.05 15:21:26 +0200, Björn Steinbrink wrote:\n> > On 2009.06.05 14:24:44 +0200, Björn Steinbrink wrote:\n> > > Hi,\n> > > \n> > > can't make any sense out of this at all:\n> > > \n> > > doener@atjola:git (master) $ git reset --hard HEAD@{1}\n> > > HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'\n> > > doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD\n> > > \n> > > doener@atjola:git (master) $ git pull\n> > > >From git://git.kernel.org/pub/scm/git/git\n> > > b11cf09..6096d75 master -> origin/master\n> > > Updating b11cf09..6096d75\n> > > Fast forward\n> > > Documentation/RelNotes-1.6.3.2.txt | 12 +++++-------\n> > > Documentation/git.txt | 7 ++++++-\n> > > contrib/completion/git-completion.bash | 12 ++++++++++--\n> > > grep.c | 6 +++++-\n> > > 4 files changed, 26 insertions(+), 11 deletions(-)\n> > > \n> > > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > > doener@atjola:git (master) $ git diff-index --name-only --cached HEAD\n> > > \n> > > *wait a minute, doing nothing*\n> > > \n> > > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > > Documentation/RelNotes-1.6.3.2.txt\n> > > Documentation/git.txt\n> > > contrib/completion/git-completion.bash\n> > > grep.c\n> > \n> > Hm, looks like this is not a git bug. Went back to 1.5.4, and even that\n> > shows the error. So I actually looked at the files, and indeed, the file\n> > in the working tree gets modified. stat(1) shows:\n> > \n> > Right after the merge:\n> > File: `grep.c'\n> > Size: 16274 Blocks: 32 IO Block: 4096 regular file\n> > Device: fd03h/64771d Inode: 5933481 Links: 1\n> > Access: (0644/-rw-r--r--) Uid: ( 1000/ doener) Gid: ( 1000/ doener)\n> > Access: 2009-06-05 15:02:14.000000000 +0200\n> > Modify: 2009-06-05 15:02:14.000000000 +0200\n> > Change: 2009-06-05 15:02:14.000000000 +0200\n> > \n> > 60 seconds later:\n> > File: `grep.c'\n> > Size: 16274 Blocks: 32 IO Block: 4096 regular file\n> > Device: fd03h/64771d Inode: 5933481 Links: 1\n> > Access: (0644/-rw-r--r--) Uid: ( 1000/ doener) Gid: ( 1000/ doener)\n> > Access: 2009-06-05 15:02:14.000000000 +0200\n> > Modify: 2009-06-05 15:02:14.000000000 +0200\n> > Change: 2009-06-05 15:02:48.000000000 +0200\n> > \n> > So the ctime got modified. I don't have any fancy indexing stuff\n> > running, and inotify doesn't see any events either while the ctime is\n> > changed.\n> > \n> > The only thing I changed lately was upgrading to 2.6.30-rc8 and going\n> > from ext3 to ext4. As the ctime change always seems to happen around 30\n> > seconds after the real change, I kind of suspect ext4 to be guilty.\n> > Ted, is that possible?\n> > \n> > FS is mounted as:\n> > /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)\n> \n> Hm, yup, seems to be ext4 related, doesn't happen on tmpfs.\n\nOK, so I tested a bit more. And my /home is really just ext3 mounted\nwith the ext4 driver:\n\nFilesystem features: has_journal resize_inode dir_index filetype\nneeds_recovery sparse_super large_file\n\nWhile my /tmp actually got the ext4 features:\nFilesystem features: has_journal ext_attr resize_inode dir_index\nfiletype needs_recovery extent sparse_super large_file uninit_bg\n\nAnd on /tmp, ctime doesn't change.\n\nPlaying around a bit more, the ctime change is easily reproducable for\nme, without waiting 30-60 seconds. Just running sync will do:\n\nrm foo # We need a new file to be created\necho 123 > foo\nstat foo\nsleep 2\nsync\nstat foo\n\n\nOn /home, that sequence shows ctime having changed after the sync, while\non /tmp, no such change happens.\n\nIs this a fault on my side? I thought I had read that you could just\nmount ext3 filesystems without any changes and benefit from delayed\nallocation and such. Hm?\n\nBjörn\n"},{"id":"115508","messageId":"20090605145508.GA6442@mit.edu","threadId":"19675","inReplyTo":"20090605132126.GB11035@atjola.homenet","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Theodore Tso","fromEmail":"tytso@mit.edu","sentAt":"2009-06-05T14:55:08Z","receivedAt":"2009-06-05T14:55:08Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Fri, Jun 05, 2009 at 03:21:26PM +0200, Björn Steinbrink wrote:\n> So the ctime got modified. I don't have any fancy indexing stuff\n> running, and inotify doesn't see any events either while the ctime is\n> changed.\n> \n> The only thing I changed lately was upgrading to 2.6.30-rc8 and going\n> from ext3 to ext4. As the ctime change always seems to happen around 30\n> seconds after the real change, I kind of suspect ext4 to be guilty.\n> Ted, is that possible?\n> \n> FS is mounted as:\n> /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)\n> \n\nI agree it sounds like it's ext4 related, but I'm not able to\nreproduce it (using 2.6.30-rc8 with the patches planned for the 2.6.31\nmerge window). This should show the problem, you were seeing, do you\nagree?\n\n {/usr/projects/linux/misc} [master]\n547% git reset --hard HEAD^\nHEAD is now at b63254c Merge branch 'drm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6\n {/usr/projects/linux/misc} [master]\n548% git pull\nUpdating b63254c..b87297f\nFast forward\n drivers/gpu/drm/i915/i915_gem.c | 3 ---\n 1 files changed, 0 insertions(+), 3 deletions(-)\n {/usr/projects/linux/misc} [master]\n549% stat drivers/gpu/drm/i915/i915_gem.c;sleep 60 ; stat drivers/gpu/drm/i915/i915_gem.c\n File: `drivers/gpu/drm/i915/i915_gem.c'\n Size: 117594\t\tBlocks: 232 IO Block: 4096 regular file\nDevice: fe00h/65024d\tInode: 2643071 Links: 1\nAccess: (0644/-rw-r--r--) Uid: (15806/ tytso) Gid: (15806/ tytso)\nAccess: 2009-06-05 10:50:05.466966433 -0400\nModify: 2009-06-05 10:50:05.466966433 -0400\nChange: 2009-06-05 10:50:05.466966433 -0400\n File: `drivers/gpu/drm/i915/i915_gem.c'\n Size: 117594\t\tBlocks: 232 IO Block: 4096 regular file\nDevice: fe00h/65024d\tInode: 2643071 Links: 1\nAccess: (0644/-rw-r--r--) Uid: (15806/ tytso) Gid: (15806/ tytso)\nAccess: 2009-06-05 10:50:05.466966433 -0400\nModify: 2009-06-05 10:50:05.466966433 -0400\nChange: 2009-06-05 10:50:05.466966433 -0400\n\nHmm, can you send me the output of dumpe2fs -h for your /home\nfilesystem?\n\n\t\t\t\t\t\t- Ted\n"},{"id":"115509","messageId":"20090605150212.GC764@atjola.homenet","threadId":"19675","inReplyTo":"20090605145508.GA6442@mit.edu","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-06-05T15:02:12Z","receivedAt":"2009-06-05T15:02:12Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.06.05 10:55:08 -0400, Theodore Tso wrote:\n> On Fri, Jun 05, 2009 at 03:21:26PM +0200, Björn Steinbrink wrote:\n> > So the ctime got modified. I don't have any fancy indexing stuff\n> > running, and inotify doesn't see any events either while the ctime is\n> > changed.\n> > \n> > The only thing I changed lately was upgrading to 2.6.30-rc8 and going\n> > from ext3 to ext4. As the ctime change always seems to happen around 30\n> > seconds after the real change, I kind of suspect ext4 to be guilty.\n> > Ted, is that possible?\n> > \n> > FS is mounted as:\n> > /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)\n> > \n> \n> I agree it sounds like it's ext4 related, but I'm not able to\n> reproduce it (using 2.6.30-rc8 with the patches planned for the 2.6.31\n> merge window). This should show the problem, you were seeing, do you\n> agree?\n\nYeah, that should do I guess. See my other mail for a simpler, less\ntime consuming way to test. And as noted in there, it seems to happen\nonly on ext3 filesystems mounted using ext4.\n\n> {/usr/projects/linux/misc} [master]\n> 547% git reset --hard HEAD^\n> HEAD is now at b63254c Merge branch 'drm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6\n> {/usr/projects/linux/misc} [master]\n> 548% git pull\n> Updating b63254c..b87297f\n> Fast forward\n> drivers/gpu/drm/i915/i915_gem.c | 3 ---\n> 1 files changed, 0 insertions(+), 3 deletions(-)\n> {/usr/projects/linux/misc} [master]\n> 549% stat drivers/gpu/drm/i915/i915_gem.c;sleep 60 ; stat drivers/gpu/drm/i915/i915_gem.c\n> File: `drivers/gpu/drm/i915/i915_gem.c'\n> Size: 117594\t\tBlocks: 232 IO Block: 4096 regular file\n> Device: fe00h/65024d\tInode: 2643071 Links: 1\n> Access: (0644/-rw-r--r--) Uid: (15806/ tytso) Gid: (15806/ tytso)\n> Access: 2009-06-05 10:50:05.466966433 -0400\n> Modify: 2009-06-05 10:50:05.466966433 -0400\n> Change: 2009-06-05 10:50:05.466966433 -0400\n> File: `drivers/gpu/drm/i915/i915_gem.c'\n> Size: 117594\t\tBlocks: 232 IO Block: 4096 regular file\n> Device: fe00h/65024d\tInode: 2643071 Links: 1\n> Access: (0644/-rw-r--r--) Uid: (15806/ tytso) Gid: (15806/ tytso)\n> Access: 2009-06-05 10:50:05.466966433 -0400\n> Modify: 2009-06-05 10:50:05.466966433 -0400\n> Change: 2009-06-05 10:50:05.466966433 -0400\n> \n> Hmm, can you send me the output of dumpe2fs -h for your /home\n> filesystem?\n\ndumpe2fs 1.41.6 (30-May-2009)\nFilesystem volume name: \nLast mounted on: \nFilesystem UUID: 4cc724c1-e366-41d7-8dee-baa6624bd04c\nFilesystem magic number: 0xEF53\nFilesystem revision #: 1 (dynamic)\nFilesystem features: has_journal resize_inode dir_index filetype\n needs_recovery sparse_super large_file\nFilesystem flags: signed_directory_hash \nDefault mount options: (none)\nFilesystem state: clean\nErrors behavior: Continue\nFilesystem OS type: Linux\nInode count: 15548416\nBlock count: 31095808\nReserved block count: 0\nFree blocks: 19016208\nFree inodes: 15221178\nFirst block: 0\nBlock size: 4096\nFragment size: 4096\nReserved GDT blocks: 1016\nBlocks per group: 32768\nFragments per group: 32768\nInodes per group: 16384\nInode blocks per group: 512\nFilesystem created: Tue May 15 19:50:48 2007\nLast mount time: Thu Jun 4 21:31:36 2009\nLast write time: Thu Jun 4 21:31:36 2009\nMount count: 17\nMaximum mount count: 21\nLast checked: Tue Mar 24 15:15:08 2009\nCheck interval: 15552000 (6 months)\nNext check after: Sun Sep 20 16:15:08 2009\nLifetime writes: 37 GB\nReserved blocks uid: 0 (user root)\nReserved blocks gid: 0 (group root)\nFirst inode: 11\nInode size: 128\nJournal inode: 8\nDefault directory hash: tea\nDirectory Hash Seed: 55e920c6-f31f-4761-b9d3-3ab558dc5746\nJournal backup: inode blocks\nJournal size: 128M\n\nThanks,\nBjörn\n"},{"id":"115512","messageId":"alpine.LFD.2.01.0906050825120.6847@localhost.localdomain","threadId":"19675","inReplyTo":"20090605132126.GB11035@atjola.homenet","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Linus Torvalds","fromEmail":"torvalds@linux-foundation.org","sentAt":"2009-06-05T15:39:24Z","receivedAt":"2009-06-05T15:39:24Z","isPatch":false,"sender":{"key":"torvalds@linux-foundation.org","avatar":"https://avatars.githubusercontent.com/u/1024025?v=4"},"body":"\n\nOn Fri, 5 Jun 2009, Björn Steinbrink wrote:\n> > \n> > *wait a minute, doing nothing*\n> > \n> > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > Documentation/RelNotes-1.6.3.2.txt\n> > Documentation/git.txt\n> > contrib/completion/git-completion.bash\n> > grep.c\n> \n> Hm, looks like this is not a git bug. Went back to 1.5.4, and even that\n> shows the error. So I actually looked at the files, and indeed, the file\n> in the working tree gets modified. stat(1) shows:\n\nYou have beagle running, don't you?\n\nIt's a piece of sh*t, and uses extended attributes to track indexing \ninformation.\n\nSo the \"wait a minute\" will just give beagle the chance to index your git \nworking tree, and update the extended attributes. That is entirely hidden \nfrom all normal filesystem stat information, *EXCEPT* it changes ctime, \nsince the inode is changed.\n\nIt's annoying as hell. Beagle is broken. It's a particularly inefficient \nway to store index information, and it is annoying that an indexing engine \nactually changes the files it indexes. \n\nYou can tell git to ignore CTIME, by using\n\n\t[core]\n\t\ttrustctime = false\n\nand now git will ignore the mess that beagle makes of things. \n\nThe other alternative is to tell beagle to behave. The beagle people \nclaims this makes things slower. Which is quite possibly true, since the \nkernel is optimized and caches things well (extended attributes in the \nfilesystem), and beagles alternative model is probably some incredibly \ncrappy indexing built on top of MySQL.\n\nYour choice.\n\n\t\tLinus\n"},{"id":"115537","messageId":"20090605161444.GD764@atjola.homenet","threadId":"19675","inReplyTo":"alpine.LFD.2.01.0906050825120.6847@localhost.localdomain","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-06-05T16:14:44Z","receivedAt":"2009-06-05T16:14:44Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.06.05 08:39:24 -0700, Linus Torvalds wrote:\n> \n> \n> On Fri, 5 Jun 2009, Björn Steinbrink wrote:\n> > > \n> > > *wait a minute, doing nothing*\n> > > \n> > > doener@atjola:git (master) $ git diff-index --name-only HEAD\n> > > Documentation/RelNotes-1.6.3.2.txt\n> > > Documentation/git.txt\n> > > contrib/completion/git-completion.bash\n> > > grep.c\n> > \n> > Hm, looks like this is not a git bug. Went back to 1.5.4, and even that\n> > shows the error. So I actually looked at the files, and indeed, the file\n> > in the working tree gets modified. stat(1) shows:\n> \n> You have beagle running, don't you?\n\nNo, nothing like that is running here, I even switched to runlevel 1\njust to make sure ;-)\n\n> It's a piece of sh*t, and uses extended attributes to track indexing \n> information.\n> \n> So the \"wait a minute\" will just give beagle the chance to index your git \n> working tree, and update the extended attributes. That is entirely hidden \n> from all normal filesystem stat information, *EXCEPT* it changes ctime, \n> since the inode is changed.\n\nNo, I'm pretty sure now that it's due to the fact that I use the ext4\nkernel driver with an ext3 fs, as running sync(1) is as good as waiting\na minute. And the problem shows up neither with tmpfs nor with a real\next4.\n\nBjörn\n"},{"id":"115548","messageId":"20090605180630.GC6442@mit.edu","threadId":"19675","inReplyTo":"20090605150212.GC764@atjola.homenet","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Theodore Tso","fromEmail":"tytso@mit.edu","sentAt":"2009-06-05T18:06:30Z","receivedAt":"2009-06-05T18:06:30Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Fri, Jun 05, 2009 at 05:02:12PM +0200, Björn Steinbrink wrote:\n> On 2009.06.05 10:55:08 -0400, Theodore Tso wrote:\n> > On Fri, Jun 05, 2009 at 03:21:26PM +0200, Björn Steinbrink wrote:\n> > > So the ctime got modified. I don't have any fancy indexing stuff\n> > > running, and inotify doesn't see any events either while the ctime is\n> > > changed.\n> > > \n> > > The only thing I changed lately was upgrading to 2.6.30-rc8 and going\n> > > from ext3 to ext4. As the ctime change always seems to happen around 30\n> > > seconds after the real change, I kind of suspect ext4 to be guilty.\n> > > Ted, is that possible?\n> > > \n> > > FS is mounted as:\n> > > /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)\n> > > \n> > \n> > I agree it sounds like it's ext4 related, but I'm not able to\n> > reproduce it (using 2.6.30-rc8 with the patches planned for the 2.6.31\n> > merge window). This should show the problem, you were seeing, do you\n> > agree?\n> \n> Yeah, that should do I guess. See my other mail for a simpler, less\n> time consuming way to test. And as noted in there, it seems to happen\n> only on ext3 filesystems mounted using ext4.\n\n> Filesystem features: has_journal resize_inode dir_index filetype\n> needs_recovery sparse_super large_file\n\nYeah, so you haven't turned on any of the ext4 filesystem features, I\nassume because you wanted to be easily go back to ext3 if you ran into\nproblems? OK, that's a good starting point.\n\nI'm guessing it's the presence or absence of one of the ext4-specific\nfilesystem features, most probably the extents feature (which is why I\nhad asked you to to send me your dumpe2fs -h output). \n\nSo the next step is to create an ext3 filesystem with a git repository\non it, and then to gradually turn on various ext4 specific features\nand see when the bug ends up getting replicated. If I had to guess\nit's the lack (or absense) of the extents feature, but I'll have to\nrun the test and find out for sure.\n\nThanks for reporting the bug. Let me see what I can find.\n\n \t \t \t \t \t - Ted\n"},{"id":"115549","messageId":"20090605183538.GE764@atjola.homenet","threadId":"19675","inReplyTo":"20090605180630.GC6442@mit.edu","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Björn Steinbrink","fromEmail":"b.steinbrink@gmx.de","sentAt":"2009-06-05T18:35:38Z","receivedAt":"2009-06-05T18:35:38Z","isPatch":false,"sender":{"key":"b.steinbrink@gmx.de","avatar":"https://avatars.githubusercontent.com/u/230962?v=4"},"body":"On 2009.06.05 14:06:30 -0400, Theodore Tso wrote:\n> On Fri, Jun 05, 2009 at 05:02:12PM +0200, Björn Steinbrink wrote:\n> > On 2009.06.05 10:55:08 -0400, Theodore Tso wrote:\n> > > On Fri, Jun 05, 2009 at 03:21:26PM +0200, Björn Steinbrink wrote:\n> > > > So the ctime got modified. I don't have any fancy indexing stuff\n> > > > running, and inotify doesn't see any events either while the ctime is\n> > > > changed.\n> > > > \n> > > > The only thing I changed lately was upgrading to 2.6.30-rc8 and going\n> > > > from ext3 to ext4. As the ctime change always seems to happen around 30\n> > > > seconds after the real change, I kind of suspect ext4 to be guilty.\n> > > > Ted, is that possible?\n> > > > \n> > > > FS is mounted as:\n> > > > /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)\n> > > > \n> > > \n> > > I agree it sounds like it's ext4 related, but I'm not able to\n> > > reproduce it (using 2.6.30-rc8 with the patches planned for the 2.6.31\n> > > merge window). This should show the problem, you were seeing, do you\n> > > agree?\n> > \n> > Yeah, that should do I guess. See my other mail for a simpler, less\n> > time consuming way to test. And as noted in there, it seems to happen\n> > only on ext3 filesystems mounted using ext4.\n> \n> > Filesystem features: has_journal resize_inode dir_index filetype\n> > needs_recovery sparse_super large_file\n> \n> Yeah, so you haven't turned on any of the ext4 filesystem features, I\n> assume because you wanted to be easily go back to ext3 if you ran into\n> problems? OK, that's a good starting point.\n\nRight.\n\n> I'm guessing it's the presence or absence of one of the ext4-specific\n> filesystem features, most probably the extents feature (which is why I\n> had asked you to to send me your dumpe2fs -h output). \n> \n> So the next step is to create an ext3 filesystem with a git repository\n> on it, and then to gradually turn on various ext4 specific features\n> and see when the bug ends up getting replicated. If I had to guess\n> it's the lack (or absense) of the extents feature, but I'll have to\n> run the test and find out for sure.\n\nYep, seems to be extents. Test script:\n\n#!bin/bash\n\ndo_test ()\n{\n\tmke2fs -j -m 1 /dev/loop0 -O $1 2>&1\n\tmount -t ext4 /dev/loop0 foo\n\tcd foo\n\tgit init \n\techo 123 > foo\n\tgit add foo\n\tgit commit -m test\n\tsleep 2\n\tsync\n\tgit diff-index --exit-code HEAD && echo \"Good: $1\" >&2 ||\n\t\techo \"Bad: $1\" >&2\n\tcd ..\n\tumount foo\n}\n\n\nfor opt in extent dir_index uninit_bg extent,dir_index extent,uninit_bg\\\n\tdir_index,uninit_bg extent,dir_index,uninit_bg\ndo\n\tdo_test \"$opt\" > /dev/null\ndone\n\nResults:\n$ sudo bash e3t.sh \nGood: extent\nBad: dir_index\nBad: uninit_bg\nGood: extent,dir_index\nGood: extent,uninit_bg\nBad: dir_index,uninit_bg\nGood: extent,dir_index,uninit_bg\n\nBjörn\n"},{"id":"115570","messageId":"20090605213824.GD6442@mit.edu","threadId":"19675","inReplyTo":"20090605183538.GE764@atjola.homenet","subject":"Re: [BUG ext4?] Working tree getting out of date \"spontaneously\"","fromName":"Theodore Tso","fromEmail":"tytso@mit.edu","sentAt":"2009-06-05T21:38:24Z","receivedAt":"2009-06-05T21:38:24Z","isPatch":false,"sender":{"key":"tytso@mit.edu","avatar":"https://avatars.githubusercontent.com/u/51416?v=4"},"body":"On Fri, Jun 05, 2009 at 08:35:38PM +0200, Björn Steinbrink wrote:\n> > So the next step is to create an ext3 filesystem with a git repository\n> > on it, and then to gradually turn on various ext4 specific features\n> > and see when the bug ends up getting replicated. If I had to guess\n> > it's the lack (or absense) of the extents feature, but I'll have to\n> > run the test and find out for sure.\n> \n> Yep, seems to be extents. Test script:\n\nOK, I see what's going on. When doing delayed allocation, and we're\nnot using extents, the call to ext4_get_blocks() which does the\nallocation ultimately ends up calling ext4_slice_branch if the inode\nis using direct/indirect blocks instead of extents.\next4_splice_branch() sets ctime. Taking out the this line in\nfs/ext4/inode.c:ext4_splice_branch() should fix things:\n\n\n\t/* We are done with atomic stuff, now do the rest of housekeeping */\n\n-\tinode->i_ctime = ext4_current_time(inode);\n\text4_mark_inode_dirty(handle, inode);\n\n\t/* had we spliced it onto indirect block? */\n\nI'm pretty sure we don't need to set i_ctime anywhere else, since we\ndon't have a similar line in the extents code and we\nfs/inode.c:file_update_time() should take care updating i_ctime where\nit needs it, but I want to take a closer look to be sure.\n\n \t \t \t \t \t - Ted\n"}]}