threads / bug / 19675

[BUG?] Working tree getting out of date "spontaneously"

Subject: [BUG?] Working tree getting out of date "spontaneously"

## tl;dr

13 messages between Jun 5, 2009 and Jun 5, 2009.

replies: 12people: 4as markdown or json

Björn Steinbrink· Jun 5, 2009, 12:24 UTC · lore
Hi,
can't make any sense out of this at all:

doener@atjola:git (master) $ git reset --hard HEAD@{1} HEAD is now at b11cf09 Merge branch 'da/pretty-tempname' doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD

doener@atjola:git (master) $ git pull
From git://git.kernel.org/pub/scm/git/git
   b11cf09..6096d75  master     -> origin/master
Updating b11cf09..6096d75
Fast forward
 Documentation/RelNotes-1.6.3.2.txt     |   12 +++++-------
 Documentation/git.txt                  |    7 ++++++-
 contrib/completion/git-completion.bash |   12 ++++++++++--
 grep.c                                 |    6 +++++-
 4 files changed, 26 insertions(+), 11 deletions(-)

doener@atjola:git (master) $ git diff-index --name-only HEAD doener@atjola:git (master) $ git diff-index --name-only --cached HEAD

*wait a minute, doing nothing*

doener@atjola:git (master) $ git diff-index --name-only HEAD Documentation/RelNotes-1.6.3.2.txt Documentation/git.txt contrib/completion/git-completion.bash grep.c

doener@atjola:git (master) $ git diff-index --name-only --cached HEAD
doener@atjola:git (master) $ git diff-index --stat HEAD
 0 files changed, 0 insertions(+), 0 deletions(-)

doener@atjola:git (master) $ git diff-index --name-only HEAD Documentation/RelNotes-1.6.3.2.txt Documentation/git.txt contrib/completion/git-completion.bash grep.c

Running "git status" seems to fix things.
Björn, confused
Robin Rosenberg· Jun 5, 2009, 13:14 UTC · re: Björn Steinbrink · lore

Re: [BUG?] Working tree getting out of date "spontaneously"

fredag 05 juni 2009 14:24:44 skrev Björn Steinbrink:
Show 45 quoted lines
> Hi,
> 
> can't make any sense out of this at all:
> 
> doener@atjola:git (master) $ git reset --hard HEAD@{1}
> HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'
> doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD
> 
> doener@atjola:git (master) $ git pull
> From git://git.kernel.org/pub/scm/git/git
>    b11cf09..6096d75  master     -> origin/master
> Updating b11cf09..6096d75
> Fast forward
>  Documentation/RelNotes-1.6.3.2.txt     |   12 +++++-------
>  Documentation/git.txt                  |    7 ++++++-
>  contrib/completion/git-completion.bash |   12 ++++++++++--
>  grep.c                                 |    6 +++++-
>  4 files changed, 26 insertions(+), 11 deletions(-)
> 
> doener@atjola:git (master) $ git diff-index --name-only HEAD
> doener@atjola:git (master) $ git diff-index --name-only --cached HEAD
> 
> *wait a minute, doing nothing*
> 
> doener@atjola:git (master) $ git diff-index --name-only HEAD
> Documentation/RelNotes-1.6.3.2.txt
> Documentation/git.txt
> contrib/completion/git-completion.bash
> grep.c
> 
> doener@atjola:git (master) $ git diff-index --name-only --cached HEAD
> 
> doener@atjola:git (master) $ git diff-index --stat HEAD
>  0 files changed, 0 insertions(+), 0 deletions(-)
> 
> doener@atjola:git (master) $ git diff-index --name-only HEAD
> Documentation/RelNotes-1.6.3.2.txt
> Documentation/git.txt
> contrib/completion/git-completion.bash
> grep.c
> 
> 
> Running "git status" seems to fix things.
> 
> Björn, confused
What file system and OS?
-- robin
Björn Steinbrink· Jun 5, 2009, 13:23 UTC · re: Robin Rosenberg · lore

Re: [BUG?] Working tree getting out of date "spontaneously"

On 2009.06.05 15:14:15 +0200, Robin Rosenberg wrote:
Show 48 quoted lines
> fredag 05 juni 2009 14:24:44 skrev Björn Steinbrink:
> > Hi,
> > 
> > can't make any sense out of this at all:
> > 
> > doener@atjola:git (master) $ git reset --hard HEAD@{1}
> > HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'
> > doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD
> > 
> > doener@atjola:git (master) $ git pull
> > From git://git.kernel.org/pub/scm/git/git
> >    b11cf09..6096d75  master     -> origin/master
> > Updating b11cf09..6096d75
> > Fast forward
> >  Documentation/RelNotes-1.6.3.2.txt     |   12 +++++-------
> >  Documentation/git.txt                  |    7 ++++++-
> >  contrib/completion/git-completion.bash |   12 ++++++++++--
> >  grep.c                                 |    6 +++++-
> >  4 files changed, 26 insertions(+), 11 deletions(-)
> > 
> > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > doener@atjola:git (master) $ git diff-index --name-only --cached HEAD
> > 
> > *wait a minute, doing nothing*
> > 
> > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > Documentation/RelNotes-1.6.3.2.txt
> > Documentation/git.txt
> > contrib/completion/git-completion.bash
> > grep.c
> > 
> > doener@atjola:git (master) $ git diff-index --name-only --cached HEAD
> > 
> > doener@atjola:git (master) $ git diff-index --stat HEAD
> >  0 files changed, 0 insertions(+), 0 deletions(-)
> > 
> > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > Documentation/RelNotes-1.6.3.2.txt
> > Documentation/git.txt
> > contrib/completion/git-completion.bash
> > grep.c
> > 
> > 
> > Running "git status" seems to fix things.
> > 
> > Björn, confused
> 
> What file system and OS?

Linux 2.6.30-rc8, ext4. ctime gets updated for no apparent reason, see the other mail I just sent.

Thanks, Björn

Björn Steinbrink· Jun 5, 2009, 13:21 UTC · re: Björn Steinbrink · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On 2009.06.05 14:24:44 +0200, Björn Steinbrink wrote:
Show 29 quoted lines
> Hi,
> 
> can't make any sense out of this at all:
> 
> doener@atjola:git (master) $ git reset --hard HEAD@{1}
> HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'
> doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD
> 
> doener@atjola:git (master) $ git pull
> >From git://git.kernel.org/pub/scm/git/git
>    b11cf09..6096d75  master     -> origin/master
> Updating b11cf09..6096d75
> Fast forward
>  Documentation/RelNotes-1.6.3.2.txt     |   12 +++++-------
>  Documentation/git.txt                  |    7 ++++++-
>  contrib/completion/git-completion.bash |   12 ++++++++++--
>  grep.c                                 |    6 +++++-
>  4 files changed, 26 insertions(+), 11 deletions(-)
> 
> doener@atjola:git (master) $ git diff-index --name-only HEAD
> doener@atjola:git (master) $ git diff-index --name-only --cached HEAD
> 
> *wait a minute, doing nothing*
> 
> doener@atjola:git (master) $ git diff-index --name-only HEAD
> Documentation/RelNotes-1.6.3.2.txt
> Documentation/git.txt
> contrib/completion/git-completion.bash
> grep.c

Hm, looks like this is not a git bug. Went back to 1.5.4, and even that shows the error. So I actually looked at the files, and indeed, the file in the working tree gets modified. stat(1) shows:

Right after the merge:
  File: `grep.c'
  Size: 16274           Blocks: 32         IO Block: 4096   regular file
Device: fd03h/64771d    Inode: 5933481     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  doener)   Gid: ( 1000/  doener)
Access: 2009-06-05 15:02:14.000000000 +0200
Modify: 2009-06-05 15:02:14.000000000 +0200
Change: 2009-06-05 15:02:14.000000000 +0200
60 seconds later:
  File: `grep.c'
  Size: 16274           Blocks: 32         IO Block: 4096   regular file
Device: fd03h/64771d    Inode: 5933481     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/  doener)   Gid: ( 1000/  doener)
Access: 2009-06-05 15:02:14.000000000 +0200
Modify: 2009-06-05 15:02:14.000000000 +0200
Change: 2009-06-05 15:02:48.000000000 +0200

So the ctime got modified. I don't have any fancy indexing stuff running, and inotify doesn't see any events either while the ctime is changed.

The only thing I changed lately was upgrading to 2.6.30-rc8 and going from ext3 to ext4. As the ctime change always seems to happen around 30 seconds after the real change, I kind of suspect ext4 to be guilty. Ted, is that possible?

FS is mounted as: /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)

Björn
Björn Steinbrink· Jun 5, 2009, 14:12 UTC · re: Björn Steinbrink · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On 2009.06.05 15:21:26 +0200, Björn Steinbrink wrote:
Show 64 quoted lines
> On 2009.06.05 14:24:44 +0200, Björn Steinbrink wrote:
> > Hi,
> > 
> > can't make any sense out of this at all:
> > 
> > doener@atjola:git (master) $ git reset --hard HEAD@{1}
> > HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'
> > doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD
> > 
> > doener@atjola:git (master) $ git pull
> > >From git://git.kernel.org/pub/scm/git/git
> >    b11cf09..6096d75  master     -> origin/master
> > Updating b11cf09..6096d75
> > Fast forward
> >  Documentation/RelNotes-1.6.3.2.txt     |   12 +++++-------
> >  Documentation/git.txt                  |    7 ++++++-
> >  contrib/completion/git-completion.bash |   12 ++++++++++--
> >  grep.c                                 |    6 +++++-
> >  4 files changed, 26 insertions(+), 11 deletions(-)
> > 
> > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > doener@atjola:git (master) $ git diff-index --name-only --cached HEAD
> > 
> > *wait a minute, doing nothing*
> > 
> > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > Documentation/RelNotes-1.6.3.2.txt
> > Documentation/git.txt
> > contrib/completion/git-completion.bash
> > grep.c
> 
> Hm, looks like this is not a git bug. Went back to 1.5.4, and even that
> shows the error. So I actually looked at the files, and indeed, the file
> in the working tree gets modified. stat(1) shows:
> 
> Right after the merge:
>   File: `grep.c'
>   Size: 16274           Blocks: 32         IO Block: 4096   regular file
> Device: fd03h/64771d    Inode: 5933481     Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/  doener)   Gid: ( 1000/  doener)
> Access: 2009-06-05 15:02:14.000000000 +0200
> Modify: 2009-06-05 15:02:14.000000000 +0200
> Change: 2009-06-05 15:02:14.000000000 +0200
> 
> 60 seconds later:
>   File: `grep.c'
>   Size: 16274           Blocks: 32         IO Block: 4096   regular file
> Device: fd03h/64771d    Inode: 5933481     Links: 1
> Access: (0644/-rw-r--r--)  Uid: ( 1000/  doener)   Gid: ( 1000/  doener)
> Access: 2009-06-05 15:02:14.000000000 +0200
> Modify: 2009-06-05 15:02:14.000000000 +0200
> Change: 2009-06-05 15:02:48.000000000 +0200
> 
> So the ctime got modified. I don't have any fancy indexing stuff
> running, and inotify doesn't see any events either while the ctime is
> changed.
> 
> The only thing I changed lately was upgrading to 2.6.30-rc8 and going
> from ext3 to ext4. As the ctime change always seems to happen around 30
> seconds after the real change, I kind of suspect ext4 to be guilty.
> Ted, is that possible?
> 
> FS is mounted as:
> /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)
Hm, yup, seems to be ext4 related, doesn't happen on tmpfs.
Björn
Björn Steinbrink· Jun 5, 2009, 14:47 UTC · re: Björn Steinbrink · lore

Re: [BUG? ext4] git working tree getting out of date "spontaneously"

On 2009.06.05 16:12:09 +0200, Björn Steinbrink wrote:
Show 67 quoted lines
> On 2009.06.05 15:21:26 +0200, Björn Steinbrink wrote:
> > On 2009.06.05 14:24:44 +0200, Björn Steinbrink wrote:
> > > Hi,
> > > 
> > > can't make any sense out of this at all:
> > > 
> > > doener@atjola:git (master) $ git reset --hard HEAD@{1}
> > > HEAD is now at b11cf09 Merge branch 'da/pretty-tempname'
> > > doener@atjola:git (master) $ git update-ref refs/remotes/origin/master HEAD
> > > 
> > > doener@atjola:git (master) $ git pull
> > > >From git://git.kernel.org/pub/scm/git/git
> > >    b11cf09..6096d75  master     -> origin/master
> > > Updating b11cf09..6096d75
> > > Fast forward
> > >  Documentation/RelNotes-1.6.3.2.txt     |   12 +++++-------
> > >  Documentation/git.txt                  |    7 ++++++-
> > >  contrib/completion/git-completion.bash |   12 ++++++++++--
> > >  grep.c                                 |    6 +++++-
> > >  4 files changed, 26 insertions(+), 11 deletions(-)
> > > 
> > > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > > doener@atjola:git (master) $ git diff-index --name-only --cached HEAD
> > > 
> > > *wait a minute, doing nothing*
> > > 
> > > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > > Documentation/RelNotes-1.6.3.2.txt
> > > Documentation/git.txt
> > > contrib/completion/git-completion.bash
> > > grep.c
> > 
> > Hm, looks like this is not a git bug. Went back to 1.5.4, and even that
> > shows the error. So I actually looked at the files, and indeed, the file
> > in the working tree gets modified. stat(1) shows:
> > 
> > Right after the merge:
> >   File: `grep.c'
> >   Size: 16274           Blocks: 32         IO Block: 4096   regular file
> > Device: fd03h/64771d    Inode: 5933481     Links: 1
> > Access: (0644/-rw-r--r--)  Uid: ( 1000/  doener)   Gid: ( 1000/  doener)
> > Access: 2009-06-05 15:02:14.000000000 +0200
> > Modify: 2009-06-05 15:02:14.000000000 +0200
> > Change: 2009-06-05 15:02:14.000000000 +0200
> > 
> > 60 seconds later:
> >   File: `grep.c'
> >   Size: 16274           Blocks: 32         IO Block: 4096   regular file
> > Device: fd03h/64771d    Inode: 5933481     Links: 1
> > Access: (0644/-rw-r--r--)  Uid: ( 1000/  doener)   Gid: ( 1000/  doener)
> > Access: 2009-06-05 15:02:14.000000000 +0200
> > Modify: 2009-06-05 15:02:14.000000000 +0200
> > Change: 2009-06-05 15:02:48.000000000 +0200
> > 
> > So the ctime got modified. I don't have any fancy indexing stuff
> > running, and inotify doesn't see any events either while the ctime is
> > changed.
> > 
> > The only thing I changed lately was upgrading to 2.6.30-rc8 and going
> > from ext3 to ext4. As the ctime change always seems to happen around 30
> > seconds after the real change, I kind of suspect ext4 to be guilty.
> > Ted, is that possible?
> > 
> > FS is mounted as:
> > /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)
> 
> Hm, yup, seems to be ext4 related, doesn't happen on tmpfs.

OK, so I tested a bit more. And my /home is really just ext3 mounted with the ext4 driver:

Filesystem features: has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file

While my /tmp actually got the ext4 features: Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent sparse_super large_file uninit_bg

And on /tmp, ctime doesn't change.

Playing around a bit more, the ctime change is easily reproducable for me, without waiting 30-60 seconds. Just running sync will do:

rm foo # We need a new file to be created echo 123 > foo stat foo sleep 2 sync stat foo

On /home, that sequence shows ctime having changed after the sync, while on /tmp, no such change happens.

Is this a fault on my side? I thought I had read that you could just mount ext3 filesystems without any changes and benefit from delayed allocation and such. Hm?

Björn
Theodore Tso· Jun 5, 2009, 14:55 UTC · re: Björn Steinbrink · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On Fri, Jun 05, 2009 at 03:21:26PM +0200, Björn Steinbrink wrote:
Show 12 quoted lines
> So the ctime got modified. I don't have any fancy indexing stuff
> running, and inotify doesn't see any events either while the ctime is
> changed.
> 
> The only thing I changed lately was upgrading to 2.6.30-rc8 and going
> from ext3 to ext4. As the ctime change always seems to happen around 30
> seconds after the real change, I kind of suspect ext4 to be guilty.
> Ted, is that possible?
> 
> FS is mounted as:
> /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)
> 

I agree it sounds like it's ext4 related, but I'm not able to reproduce it (using 2.6.30-rc8 with the patches planned for the 2.6.31 merge window). This should show the problem, you were seeing, do you agree?

<tytso@closure> {/usr/projects/linux/misc}  [master]
547% git reset --hard HEAD^
HEAD is now at b63254c Merge branch 'drm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6
<tytso@closure> {/usr/projects/linux/misc}  [master]
548% git pull
Updating b63254c..b87297f
Fast forward
 drivers/gpu/drm/i915/i915_gem.c |    3 ---
 1 files changed, 0 insertions(+), 3 deletions(-)
<tytso@closure> {/usr/projects/linux/misc}  [master]
549% stat drivers/gpu/drm/i915/i915_gem.c;sleep 60 ; stat drivers/gpu/drm/i915/i915_gem.c
  File: `drivers/gpu/drm/i915/i915_gem.c'
  Size: 117594		Blocks: 232        IO Block: 4096   regular file
Device: fe00h/65024d	Inode: 2643071     Links: 1
Access: (0644/-rw-r--r--)  Uid: (15806/   tytso)   Gid: (15806/   tytso)
Access: 2009-06-05 10:50:05.466966433 -0400
Modify: 2009-06-05 10:50:05.466966433 -0400
Change: 2009-06-05 10:50:05.466966433 -0400
  File: `drivers/gpu/drm/i915/i915_gem.c'
  Size: 117594		Blocks: 232        IO Block: 4096   regular file
Device: fe00h/65024d	Inode: 2643071     Links: 1
Access: (0644/-rw-r--r--)  Uid: (15806/   tytso)   Gid: (15806/   tytso)
Access: 2009-06-05 10:50:05.466966433 -0400
Modify: 2009-06-05 10:50:05.466966433 -0400
Change: 2009-06-05 10:50:05.466966433 -0400

Hmm, can you send me the output of dumpe2fs -h for your /home filesystem?

						- Ted
Björn Steinbrink· Jun 5, 2009, 15:02 UTC · re: Theodore Tso · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On 2009.06.05 10:55:08 -0400, Theodore Tso wrote:
Show 18 quoted lines
> On Fri, Jun 05, 2009 at 03:21:26PM +0200, Björn Steinbrink wrote:
> > So the ctime got modified. I don't have any fancy indexing stuff
> > running, and inotify doesn't see any events either while the ctime is
> > changed.
> > 
> > The only thing I changed lately was upgrading to 2.6.30-rc8 and going
> > from ext3 to ext4. As the ctime change always seems to happen around 30
> > seconds after the real change, I kind of suspect ext4 to be guilty.
> > Ted, is that possible?
> > 
> > FS is mounted as:
> > /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)
> > 
> 
> I agree it sounds like it's ext4 related, but I'm not able to
> reproduce it (using 2.6.30-rc8 with the patches planned for the 2.6.31
> merge window).  This should show the problem, you were seeing, do you
> agree?

Yeah, that should do I guess. See my other mail for a simpler, less time consuming way to test. And as noted in there, it seems to happen only on ext3 filesystems mounted using ext4.

Show 28 quoted lines
> <tytso@closure> {/usr/projects/linux/misc}  [master]
> 547% git reset --hard HEAD^
> HEAD is now at b63254c Merge branch 'drm-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/airlied/drm-2.6
> <tytso@closure> {/usr/projects/linux/misc}  [master]
> 548% git pull
> Updating b63254c..b87297f
> Fast forward
>  drivers/gpu/drm/i915/i915_gem.c |    3 ---
>  1 files changed, 0 insertions(+), 3 deletions(-)
> <tytso@closure> {/usr/projects/linux/misc}  [master]
> 549% stat drivers/gpu/drm/i915/i915_gem.c;sleep 60 ; stat drivers/gpu/drm/i915/i915_gem.c
>   File: `drivers/gpu/drm/i915/i915_gem.c'
>   Size: 117594		Blocks: 232        IO Block: 4096   regular file
> Device: fe00h/65024d	Inode: 2643071     Links: 1
> Access: (0644/-rw-r--r--)  Uid: (15806/   tytso)   Gid: (15806/   tytso)
> Access: 2009-06-05 10:50:05.466966433 -0400
> Modify: 2009-06-05 10:50:05.466966433 -0400
> Change: 2009-06-05 10:50:05.466966433 -0400
>   File: `drivers/gpu/drm/i915/i915_gem.c'
>   Size: 117594		Blocks: 232        IO Block: 4096   regular file
> Device: fe00h/65024d	Inode: 2643071     Links: 1
> Access: (0644/-rw-r--r--)  Uid: (15806/   tytso)   Gid: (15806/   tytso)
> Access: 2009-06-05 10:50:05.466966433 -0400
> Modify: 2009-06-05 10:50:05.466966433 -0400
> Change: 2009-06-05 10:50:05.466966433 -0400
> 
> Hmm, can you send me the output of dumpe2fs -h for your /home
> filesystem?
dumpe2fs 1.41.6 (30-May-2009)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          4cc724c1-e366-41d7-8dee-baa6624bd04c
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal resize_inode dir_index filetype
                          needs_recovery sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              15548416
Block count:              31095808
Reserved block count:     0
Free blocks:              19016208
Free inodes:              15221178
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1016
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16384
Inode blocks per group:   512
Filesystem created:       Tue May 15 19:50:48 2007
Last mount time:          Thu Jun  4 21:31:36 2009
Last write time:          Thu Jun  4 21:31:36 2009
Mount count:              17
Maximum mount count:      21
Last checked:             Tue Mar 24 15:15:08 2009
Check interval:           15552000 (6 months)
Next check after:         Sun Sep 20 16:15:08 2009
Lifetime writes:          37 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      55e920c6-f31f-4761-b9d3-3ab558dc5746
Journal backup:           inode blocks
Journal size:             128M

Thanks, Björn

Theodore Tso· Jun 5, 2009, 18:06 UTC · re: Björn Steinbrink · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On Fri, Jun 05, 2009 at 05:02:12PM +0200, Björn Steinbrink wrote:
Show 23 quoted lines
> On 2009.06.05 10:55:08 -0400, Theodore Tso wrote:
> > On Fri, Jun 05, 2009 at 03:21:26PM +0200, Björn Steinbrink wrote:
> > > So the ctime got modified. I don't have any fancy indexing stuff
> > > running, and inotify doesn't see any events either while the ctime is
> > > changed.
> > > 
> > > The only thing I changed lately was upgrading to 2.6.30-rc8 and going
> > > from ext3 to ext4. As the ctime change always seems to happen around 30
> > > seconds after the real change, I kind of suspect ext4 to be guilty.
> > > Ted, is that possible?
> > > 
> > > FS is mounted as:
> > > /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)
> > > 
> > 
> > I agree it sounds like it's ext4 related, but I'm not able to
> > reproduce it (using 2.6.30-rc8 with the patches planned for the 2.6.31
> > merge window).  This should show the problem, you were seeing, do you
> > agree?
> 
> Yeah, that should do I guess. See my other mail for a simpler, less
> time consuming way to test. And as noted in there, it seems to happen
> only on ext3 filesystems mounted using ext4.
> Filesystem features:      has_journal resize_inode dir_index filetype
>                           needs_recovery sparse_super large_file

Yeah, so you haven't turned on any of the ext4 filesystem features, I assume because you wanted to be easily go back to ext3 if you ran into problems? OK, that's a good starting point.

I'm guessing it's the presence or absence of one of the ext4-specific filesystem features, most probably the extents feature (which is why I had asked you to to send me your dumpe2fs -h output).

So the next step is to create an ext3 filesystem with a git repository on it, and then to gradually turn on various ext4 specific features and see when the bug ends up getting replicated. If I had to guess it's the lack (or absense) of the extents feature, but I'll have to run the test and find out for sure.

Thanks for reporting the bug.  Let me see what I can find.
       	   	     	       	      	  - Ted
Björn Steinbrink· Jun 5, 2009, 18:35 UTC · re: Theodore Tso · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On 2009.06.05 14:06:30 -0400, Theodore Tso wrote:
Show 31 quoted lines
> On Fri, Jun 05, 2009 at 05:02:12PM +0200, Björn Steinbrink wrote:
> > On 2009.06.05 10:55:08 -0400, Theodore Tso wrote:
> > > On Fri, Jun 05, 2009 at 03:21:26PM +0200, Björn Steinbrink wrote:
> > > > So the ctime got modified. I don't have any fancy indexing stuff
> > > > running, and inotify doesn't see any events either while the ctime is
> > > > changed.
> > > > 
> > > > The only thing I changed lately was upgrading to 2.6.30-rc8 and going
> > > > from ext3 to ext4. As the ctime change always seems to happen around 30
> > > > seconds after the real change, I kind of suspect ext4 to be guilty.
> > > > Ted, is that possible?
> > > > 
> > > > FS is mounted as:
> > > > /dev/mapper/vg0-home on /home type ext4 (rw,noatime,nodiratime,barrier=0)
> > > > 
> > > 
> > > I agree it sounds like it's ext4 related, but I'm not able to
> > > reproduce it (using 2.6.30-rc8 with the patches planned for the 2.6.31
> > > merge window).  This should show the problem, you were seeing, do you
> > > agree?
> > 
> > Yeah, that should do I guess. See my other mail for a simpler, less
> > time consuming way to test. And as noted in there, it seems to happen
> > only on ext3 filesystems mounted using ext4.
> 
> > Filesystem features:      has_journal resize_inode dir_index filetype
> >                           needs_recovery sparse_super large_file
> 
> Yeah, so you haven't turned on any of the ext4 filesystem features, I
> assume because you wanted to be easily go back to ext3 if you ran into
> problems?  OK, that's a good starting point.
Right.
Show 9 quoted lines
> I'm guessing it's the presence or absence of one of the ext4-specific
> filesystem features, most probably the extents feature (which is why I
> had asked you to to send me your dumpe2fs -h output).  
> 
> So the next step is to create an ext3 filesystem with a git repository
> on it, and then to gradually turn on various ext4 specific features
> and see when the bug ends up getting replicated.  If I had to guess
> it's the lack (or absense) of the extents feature, but I'll have to
> run the test and find out for sure.
Yep, seems to be extents. Test script:
#!bin/bash
do_test ()
{
	mke2fs -j -m 1 /dev/loop0 -O $1 2>&1
	mount -t ext4 /dev/loop0 foo
	cd foo
	git init 
	echo 123 > foo
	git add foo
	git commit -m test
	sleep 2
	sync
	git diff-index --exit-code HEAD && echo "Good: $1" >&2 ||
		echo "Bad: $1" >&2
	cd ..
	umount foo
}
for opt in extent dir_index uninit_bg extent,dir_index extent,uninit_bg\
	dir_index,uninit_bg extent,dir_index,uninit_bg
do
	do_test "$opt" > /dev/null
done
Results:
$ sudo bash e3t.sh 
Good: extent
Bad: dir_index
Bad: uninit_bg
Good: extent,dir_index
Good: extent,uninit_bg
Bad: dir_index,uninit_bg
Good: extent,dir_index,uninit_bg
Björn
Theodore Tso· Jun 5, 2009, 21:38 UTC · re: Björn Steinbrink · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On Fri, Jun 05, 2009 at 08:35:38PM +0200, Björn Steinbrink wrote:
Show 7 quoted lines
> > So the next step is to create an ext3 filesystem with a git repository
> > on it, and then to gradually turn on various ext4 specific features
> > and see when the bug ends up getting replicated.  If I had to guess
> > it's the lack (or absense) of the extents feature, but I'll have to
> > run the test and find out for sure.
> 
> Yep, seems to be extents. Test script:

OK, I see what's going on. When doing delayed allocation, and we're not using extents, the call to ext4_get_blocks() which does the allocation ultimately ends up calling ext4_slice_branch if the inode is using direct/indirect blocks instead of extents. ext4_splice_branch() sets ctime. Taking out the this line in fs/ext4/inode.c:ext4_splice_branch() should fix things:

	/* We are done with atomic stuff, now do the rest of housekeeping */
-	inode->i_ctime = ext4_current_time(inode);
	ext4_mark_inode_dirty(handle, inode);
	/* had we spliced it onto indirect block? */

I'm pretty sure we don't need to set i_ctime anywhere else, since we don't have a similar line in the extents code and we fs/inode.c:file_update_time() should take care updating i_ctime where it needs it, but I want to take a closer look to be sure.

   	     	   	   	  	 - Ted
Linus Torvalds· Jun 5, 2009, 15:39 UTC · re: Björn Steinbrink · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On Fri, 5 Jun 2009, Björn Steinbrink wrote:
Show 12 quoted lines
> > 
> > *wait a minute, doing nothing*
> > 
> > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > Documentation/RelNotes-1.6.3.2.txt
> > Documentation/git.txt
> > contrib/completion/git-completion.bash
> > grep.c
> 
> Hm, looks like this is not a git bug. Went back to 1.5.4, and even that
> shows the error. So I actually looked at the files, and indeed, the file
> in the working tree gets modified. stat(1) shows:
You have beagle running, don't you?

It's a piece of sh*t, and uses extended attributes to track indexing information.

So the "wait a minute" will just give beagle the chance to index your git working tree, and update the extended attributes. That is entirely hidden from all normal filesystem stat information, *EXCEPT* it changes ctime, since the inode is changed.

It's annoying as hell. Beagle is broken. It's a particularly inefficient way to store index information, and it is annoying that an indexing engine actually changes the files it indexes.

You can tell git to ignore CTIME, by using
	[core]
		trustctime = false
and now git will ignore the mess that beagle makes of things. 

The other alternative is to tell beagle to behave. The beagle people claims this makes things slower. Which is quite possibly true, since the kernel is optimized and caches things well (extended attributes in the filesystem), and beagles alternative model is probably some incredibly crappy indexing built on top of MySQL.

Your choice.
		Linus
Björn Steinbrink· Jun 5, 2009, 16:14 UTC · re: Linus Torvalds · lore

Re: [BUG ext4?] Working tree getting out of date "spontaneously"

On 2009.06.05 08:39:24 -0700, Linus Torvalds wrote:
Show 17 quoted lines
> 
> 
> On Fri, 5 Jun 2009, Björn Steinbrink wrote:
> > > 
> > > *wait a minute, doing nothing*
> > > 
> > > doener@atjola:git (master) $ git diff-index --name-only HEAD
> > > Documentation/RelNotes-1.6.3.2.txt
> > > Documentation/git.txt
> > > contrib/completion/git-completion.bash
> > > grep.c
> > 
> > Hm, looks like this is not a git bug. Went back to 1.5.4, and even that
> > shows the error. So I actually looked at the files, and indeed, the file
> > in the working tree gets modified. stat(1) shows:
> 
> You have beagle running, don't you?

No, nothing like that is running here, I even switched to runlevel 1 just to make sure ;-)

Show 7 quoted lines
> It's a piece of sh*t, and uses extended attributes to track indexing 
> information.
> 
> So the "wait a minute" will just give beagle the chance to index your git 
> working tree, and update the extended attributes. That is entirely hidden 
> from all normal filesystem stat information, *EXCEPT* it changes ctime, 
> since the inode is changed.

No, I'm pretty sure now that it's due to the fact that I use the ext4 kernel driver with an ext3 fs, as running sync(1) is as good as waiting a minute. And the problem shows up neither with tmpfs nor with a real ext4.

Björn

← back to recent threads