Re: [PATCH 00/23] [RFC] Builtin FSMonitor Feature

From: Jeff Hostetler <git@jeffhostetler.com>
Date: Apr 20, 2021, 19:13 UTC
Message-ID: <e5035fa3-4218-209a-28e7-7c1b57d7f388@jeffhostetler.com>
In-Reply-To: <nycvar.QRO.7.76.6.2104201646210.54@tvgsbejvaqbjf.bet>

On 4/20/21 11:27 AM, Johannes Schindelin wrote:

> Hi Junio,

...

Show 17 quoted lines

>>> This RFC version includes support for Windows and MacOS file system events.
>>> A Linux version will be submitted in a later patch series.
> 
> I guess this is another reason why this patch series did not see many
> reviews: the lack of a Linux backend. And I fear that the statement "A
> Linux version will be submitted in a later patch series" is a bit strong,
> given that my original implementation of that backend does not really do
> its job well: it uses `inotify` and therefore requires one handle _per
> directory_, which in turn drains the number of file handles rather quickly
> when your worktree has many directories. Meaning: It fails todoes not work in the
> massive worktrees for which it was intended.
> 
> Now, I heard rumors that there is a saner way to monitor directory trees
> in recent Linux kernel versions (Jeff, can you fill in where I am
> blanking?) and it might be a good idea to solicit volunteers to tackle
> this backend, so that the Linux-leaning crowd on this here mailing list
> is interested a bit more?

Yes, I removed the early inotify-based version because the kernel limits the number of inotify handles to 8k (at least on my Mint box) and that is a global limit -- shared by any process wanting to use inotify. The first monorepo that I tried it on had 120K directories in my sparse checkout...

I'm told there is a newer "fanotify" facility available in newer Linux kernels that behaves more like Windows and MacOS and handles subdirectories. I intend to jump into that shortly (unless someone is already familiar with fanotify and and wants to try it).

Jeff