Re: inotify to minimize stat() calls

Duy Nguyen wrote:

Clever hack.  I didn't know that there was a switch called
core.ignoreStat which will disable automatic lstat() calls altogether.
 So, Finn advises that we set this switch and run igit instead of git.
 There's a git-inotify-daemon which runs inotifywait with -m forever,
updating a modified_files hash.  When it is sent a TERM from igit
(which is what happens immediately upon execution), it writes all this
collected information about modified files to a named pipe that igit
passes to it.  igit then does a git update-index --assume-unchained
--stdin to read the data from the pipe.  Towards the end of its life,
igit starts up a fresh git-inotify-daemon for future invocations.

Finn notes in the commit message that it offers no speedup, because .gitignore files in every directory still have to be read. I think this is silly: we really should be caching .gitignore, and touching it only when lstat() reports that the file has changed.

As far as a real implementation that we'd want to merge into git.git is concerned, I have a few comments: Running multiple daemons on-the-fly for monitoring filesystem changes is not elegant at all. Keeping track of the state of so many loose daemons is a hard problem: how do we ensure any semblance of reliability without that? Systemd is a very big improvement over the legacy of a hundred loose shell scripts that SysVInit demanded. It monitors and babysits daemons; it uses cgroups to even kill misbehaving daemons. I can inspect running daemons at any time, and have a uniform way to start/ stop/ restart them.

Okay, now you're asking me to consider a system-wide daemon independent of systemd. It has to run with root privileges so it has access to everyone's repositories, which means that people have to trust it beyond doubt. What does it do? It has a generic API to watch filesystem paths and report events over an IP socket. Do you think that this will only be useful to git? Every other version control system (and presumably many other pieces of software) will want to use it. One huge downside I see of making this part of systemd is Ubuntu. They've decided not to use systemd for some unfathomable reason.

Really, the elephant in the room right now seems to be .gitignore. Until that is fixed, there is really no use of writing this inotify daemon, no? Can someone enlighten me on how exactly .gitignore files are processed?

The thread you linked me to is a 2010 email, and now it's 2013. We've been silent about inotify for three years?

Thanks for your inputs, Duy.