threads / discuss / 58091

Option to allow fsmonitor to run against repos on network file systems

Subject: Option to allow fsmonitor to run against repos on network file systems

## tl;dr

6 messages between Jun 30, 2022 and Aug 8, 2022.

replies: 5people: 3as markdown or json

Eric D· Jun 30, 2022, 17:11 UTC · lore

I can appreciate the concerns expressed here: https://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200

However, in my environment, our file servers are very capable and have the requisite support. It would be great if there was an option to override this check and allow fsmonitor to operate against network filesystems.

-Eric
Jeff Hostetler· Jul 1, 2022, 13:32 UTC · re: Eric D · lore

Re: Option to allow fsmonitor to run against repos on network file systems

On 6/30/22 1:11 PM, Eric D wrote:
Show 7 quoted lines
> I can appreciate the concerns expressed here:
> https://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200
> 
> However, in my environment, our file servers are very capable and have
> the requisite support. It would be great if there was an option to
> override this check and allow fsmonitor to operate against network
> filesystems.

Yeah, I was just being cautious. I probably should have also added concerns on the remote system being an actual Windows server or a non-Windows host running SAMBA. There were just too many combinations for me to be comfortable enabling it by default (on the initial release, at least).

Also, the ReadDirectoryChangesW() API limits the buffer size to 64k for remote handles (because of protocol limitations), so there _may_ be more of an opportunity for dropped events on very busy remote file systems. (I never saw any dropped events in my testing (without intentionally breaking things), but it is a possible concern, so again, caution and safety...) And I do handle dropped events and force a resync and send the client a "trivial" response (so it must do a regular scan), so output is still correct, but slower.

Having said all of that, I did do lots of testing and never had an issue with remote drives actually working correctly, so I think it'd be fine allow a config setting to optionally allow it. I just didn't want to clutter up things in advance if no one actually wanted to use it on remote file systems.

I think it would be fine to have a "fsmonitor.allowRemote" or "fsmonitor.allowWindowsRemote" config setting and default them to false for now. Or until we learn which combinations of remote mounts are safe and/or problematic.

Jeff
Junio C Hamano· Jul 1, 2022, 18:41 UTC · re: Jeff Hostetler · lore

Re: Option to allow fsmonitor to run against repos on network file systems

Jeff Hostetler <git@jeffhostetler.com> writes:
Show 10 quoted lines
> Having said all of that, I did do lots of testing and never had an
> issue with remote drives actually working correctly, so I think it'd
> be fine allow a config setting to optionally allow it.  I just didn't
> want to clutter up things in advance if no one actually wanted to
> use it on remote file systems.
>
> I think it would be fine to have a "fsmonitor.allowRemote" or
> "fsmonitor.allowWindowsRemote" config setting and default them to false
> for now.  Or until we learn which combinations of remote mounts are
> safe and/or problematic.

How about getting rid of "is this remote?" check altogether (which presumably would simplify the logic) and make it totally up to the user of the repository? fsmonitor.disableInRepository that is set in ~/.gitignore and lists the paths to the repositories (like safe.directory does), for which fsmonitor gets disabled, may be a handy mechanism to set up the default (and it can be re-enabled with per-repository core.fsmonitor).

Eric D· Jul 1, 2022, 19:15 UTC · re: Junio C Hamano · lore

Re: Option to allow fsmonitor to run against repos on network file systems

One more possibility:

Leave the check, but make it a warning if there are still concerns about running fsmonitor against network file systems. Maybe also provide an option to suppress the warning? Not that much different from having "fsmonitor.allowRemote" I suppose other than by default fsmonitor would "just work" for network mounts.

On Fri, Jul 1, 2022 at 2:41 PM Junio C Hamano <gitster@pobox.com> wrote:
Show 23 quoted lines
>
> Jeff Hostetler <git@jeffhostetler.com> writes:
>
> > Having said all of that, I did do lots of testing and never had an
> > issue with remote drives actually working correctly, so I think it'd
> > be fine allow a config setting to optionally allow it.  I just didn't
> > want to clutter up things in advance if no one actually wanted to
> > use it on remote file systems.
> >
> > I think it would be fine to have a "fsmonitor.allowRemote" or
> > "fsmonitor.allowWindowsRemote" config setting and default them to false
> > for now.  Or until we learn which combinations of remote mounts are
> > safe and/or problematic.
>
> How about getting rid of "is this remote?" check altogether (which
> presumably would simplify the logic) and make it totally up to the
> user of the repository?  fsmonitor.disableInRepository that is set
> in ~/.gitignore and lists the paths to the repositories (like
> safe.directory does), for which fsmonitor gets disabled, may be a
> handy mechanism to set up the default (and it can be re-enabled with
> per-repository core.fsmonitor).
>
>
Eric D· Aug 1, 2022, 18:35 UTC · re: Eric D · lore

Re: Option to allow fsmonitor to run against repos on network file systems

On Fri, Jul 1, 2022 at 3:15 PM Eric D <eric.decosta@gmail.com> wrote:
Show 33 quoted lines
>
> One more possibility:
>
> Leave the check, but make it a warning if there are still concerns
> about running fsmonitor against network file systems. Maybe also
> provide an option to suppress the warning? Not that much different
> from having "fsmonitor.allowRemote" I suppose other than by default
> fsmonitor would "just work" for network mounts.
>
> On Fri, Jul 1, 2022 at 2:41 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Jeff Hostetler <git@jeffhostetler.com> writes:
> >
> > > Having said all of that, I did do lots of testing and never had an
> > > issue with remote drives actually working correctly, so I think it'd
> > > be fine allow a config setting to optionally allow it.  I just didn't
> > > want to clutter up things in advance if no one actually wanted to
> > > use it on remote file systems.
> > >
> > > I think it would be fine to have a "fsmonitor.allowRemote" or
> > > "fsmonitor.allowWindowsRemote" config setting and default them to false
> > > for now.  Or until we learn which combinations of remote mounts are
> > > safe and/or problematic.
> >
> > How about getting rid of "is this remote?" check altogether (which
> > presumably would simplify the logic) and make it totally up to the
> > user of the repository?  fsmonitor.disableInRepository that is set
> > in ~/.gitignore and lists the paths to the repositories (like
> > safe.directory does), for which fsmonitor gets disabled, may be a
> > handy mechanism to set up the default (and it can be re-enabled with
> > per-repository core.fsmonitor).
> >
> >

After modifying the code and playing with it for a bit, I think simply removing the check (but still being able to report on it if tracing is enabled) is the way to go. Keep it simple and if new use cases arrive, deal with them then.

-Eric
Eric D· Aug 8, 2022, 21:58 UTC · re: Jeff Hostetler · lore

Re: Option to allow fsmonitor to run against repos on network file systems

On Fri, Jul 1, 2022 at 9:32 AM Jeff Hostetler <git@jeffhostetler.com> wrote:
Show 41 quoted lines
>
>
>
> On 6/30/22 1:11 PM, Eric D wrote:
> > I can appreciate the concerns expressed here:
> > https://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200
> >
> > However, in my environment, our file servers are very capable and have
> > the requisite support. It would be great if there was an option to
> > override this check and allow fsmonitor to operate against network
> > filesystems.
>
> Yeah, I was just being cautious.  I probably should have also added
> concerns on the remote system being an actual Windows server or a
> non-Windows host running SAMBA.  There were just too many combinations
> for me to be comfortable enabling it by default (on the initial
> release, at least).
>
> Also, the ReadDirectoryChangesW() API limits the buffer size to 64k
> for remote handles (because of protocol limitations), so there _may_
> be more of an opportunity for dropped events on very busy remote file
> systems.  (I never saw any dropped events in my testing (without
> intentionally breaking things), but it is a possible concern, so again,
> caution and safety...)  And I do handle dropped events and force a
> resync and send the client a "trivial" response (so it must do a regular
> scan), so output is still correct, but slower.
>
>
> Having said all of that, I did do lots of testing and never had an
> issue with remote drives actually working correctly, so I think it'd
> be fine allow a config setting to optionally allow it.  I just didn't
> want to clutter up things in advance if no one actually wanted to
> use it on remote file systems.
>
>
> I think it would be fine to have a "fsmonitor.allowRemote" or
> "fsmonitor.allowWindowsRemote" config setting and default them to false
> for now.  Or until we learn which combinations of remote mounts are
> safe and/or problematic.
>
> Jeff
OK, based on this and other conversations, I have implemented the following:
1. Introduced a new config setting, "fsmonitor.allowRemote"
"fsmonitor.allowRemote" has a default value of false. Setting it to
true overrides
fsmonitor's default behavior of rejecting network-mounted repos.
2. Restricted allowing remote repos to Windows clients using SMB
If the client is not using SMB and using a network path, then
fsmonitor will reject
the repo path regardless of the value of "fsmonitor.allowRemote"
-Eric

← back to recent threads