threads / discuss / 48147

core.fsmonitor always perform rescans

Subject: core.fsmonitor always perform rescans

## tl;dr

4 messages between Mar 26, 2018 and Apr 11, 2018.

replies: 3people: 3as markdown or json

Tatsuyuki Ishi· Mar 26, 2018, 04:41 UTC · lore

Hello,

I'm facing issue with core.fsmonitor.

I'm currently using the provided watchman hook, but it doesn't seem to record the fact that it has queried the fsmonitor backend, and as a result the timestamp passed to the hook doesn't seem to change.

As it always pass a timestamp before watchman has crawled the directories, watchman will always return all files inside the directory. This happens everytime I run a git command, resulting in slowness.

Is the timestamp not being updated an intended behavior, or is this a bug?

Tatsuyuki Ishi

Johannes Schindelin· Mar 30, 2018, 10:28 UTC · re: Tatsuyuki Ishi · lore

Re: core.fsmonitor always perform rescans

Hi,

On Mon, 26 Mar 2018, Tatsuyuki Ishi wrote:

Show 12 quoted lines

> I'm facing issue with core.fsmonitor.
> 
> I'm currently using the provided watchman hook, but it doesn't seem to
> record the fact that it has queried the fsmonitor backend, and as a
> result the timestamp passed to the hook doesn't seem to change.
> 
> As it always pass a timestamp before watchman has crawled the
> directories, watchman will always return all files inside the
> directory. This happens everytime I run a git command, resulting in
> slowness.
> 
> Is the timestamp not being updated an intended behavior, or is this a bug?

There are some bug fixes pending, and IIRC Alex was even working on some more patches that are in an intermediate stage. Alex, could you share them with us (even in their unfinished state, maybe others can help)?

Ciao, Johannes

Ben Peart· Apr 10, 2018, 14:34 UTC · re: Tatsuyuki Ishi · lore

Re: core.fsmonitor always perform rescans

On 3/26/2018 12:41 AM, Tatsuyuki Ishi wrote:

Show 15 quoted lines

> Hello,
> 
> I'm facing issue with core.fsmonitor.
> 
> I'm currently using the provided watchman hook, but it doesn't seem to
> record the fact that it has queried the fsmonitor backend, and as a
> result the timestamp passed to the hook doesn't seem to change.
> 
> As it always pass a timestamp before watchman has crawled the
> directories, watchman will always return all files inside the
> directory. This happens everytime I run a git command, resulting in
> slowness.
> 
> Is the timestamp not being updated an intended behavior, or is this a bug?
>

As a performance optimization, the fsmonitor code doesn't flag the index as dirty and force it to be written out with every command. Can you try performing a git operation (add, rm, commit, etc) that will write out an updated index and see if that fixes the issue you're seeing?

I'm considering adding a special case to force the index to be written out the first time fsmonitor is invoked and am interested to know if this would have avoided the issue you are seeing.

Thanks!

> Tatsuyuki Ishi
>

Tatsuyuki Ishi· Apr 11, 2018, 08:18 UTC · re: Ben Peart · lore

Re: core.fsmonitor always perform rescans

2018-04-10 23:34 GMT+09:00 Ben Peart <peartben@gmail.com>:

> As a performance optimization, the fsmonitor code doesn't flag the index as
> dirty and force it to be written out with every command.  Can you try
> performing a git operation (add, rm, commit, etc) that will write out an
> updated index and see if that fixes the issue you're seeing?

Yeah, that resolves the issue. Though the repo I'm working on uses submodules, so doing this in each of them isn't a easy work.

> I'm considering adding a special case to force the index to be written out
> the first time fsmonitor is invoked and am interested to know if this would
> have avoided the issue you are seeing.

Yes please. And maybe we should also flush the index when the script returns '/', which means all files?