Re: Strange "beagle" interaction..

From: J. Bruce Fields <bfields@fieldses.org>
Date: Nov 13, 2007, 21:50 UTC
Message-ID: <20071113215047.GG22590@fieldses.org>
In-Reply-To: <9e4733910711131344t381b939dg47b5e078c52be3b2@mail.gmail.com>

On Tue, Nov 13, 2007 at 04:44:33PM -0500, Jon Smirl wrote:

Show 14 quoted lines

> On 11/13/07, Linus Torvalds <torvalds@linux-foundation.org> wrote:
> >
> >
> > On Tue, 13 Nov 2007, J. Bruce Fields wrote:
> > >
> > > Last I ran across this, I believe I found it was adding extended
> > > attributes to the file.
> >
> > Yeah, I just straced it and found the same thing. It's saving fingerprints
> > and mtimes to files in the extended attributes.
> 
> Things like Beagle need a guaranteed log of global inotify events.
> That would let them efficiently find changes made since the last time
> they updated their index.

Wouldn't a simple change-attribute get you most of the way there? All you need is a number that's guaranteed to increase any time a file is updated.

Lacking that, git's current approach (snapshot all the stat data, then look closer at any files that appear to have been touched within a second of the stat) seems pretty sensible.

--b.

Show 14 quoted lines

> Right now every time Beagle starts it hasn't got a clue what has
> changed in the file system since it was last run. This forces Beagle
> to rescan the entire filesystem every time it is started. The xattrs
> are used as cache to reduce this load somewhat.
> 
> A better solution would be for the kernel to log inotify events to
> disk in a manner that survives reboots. When Beagle starts it would
> locate its last checkpoint and then process the logged inotify events
> from that time forward. This inotify logging needs to be bullet proof
> or it will mess up your Beagle index.
> 
> Logged files systems already contain the logged inotify data (in their
> own internal form). There's just no universal API for retrieving it in
> a file system independent manner.